ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Striped pool configuration
Date Thu, 15 Dec 2016 23:33:52 GMT
Yakov, thanks.

Here are some thoughts from the Son of the Earth :)

> Another reason for not switching to striped pool completely is that stripes
> are good for cache updates and single or batched get operations since they
> reduce inter-thread communication and synchronization to minimum

How did the striped pool show itself for transactional updates that involve 2 phase commit?
Any benefits or degradation?

>  however I
> think that we should run, for example, SQL queries in old fashioned pools -
> first, they can fallback to scanning random data, second, we pause cache
> updates in some partition for query execution time.


Here we should decide what’s more important for us - throughout or latency. If to execute
SQL queries in the striped pool using multiple partitioned threads then, for sure, it will
affect latency of other operations that are sitting in the queue waiting for their time to
be processed but, on the other hand, the overall throughput of the platform should be improved
because the operations will be less halted all the time by synchronization needs.

VoltDB decided to go for with the throughput while our current architecture is mostly latency
based.

—
Denis

> On Dec 15, 2016, at 11:22 AM, Yakov Zhdanov <yzhdanov@gridgain.com> wrote:
> 
> Denis, there was no discussion on dev list. Do not worry, Son of the Earth,
> you are all right :) I thought I would send letter a bit later when I
> finish the full scope of optimizations I have in plan.
> 
> Currently, I use striped pool only for cache messages that are targeted to
> some defined partition - cache update benchmarks showed very good growth in
> throughput.
> 
> I tried to switch public and system pool to striped mode, but I get a lot
> of hangs on TC - sometimes due to issues in tests sometimes due to issues
> in Ignite. For example, under some circumstances we can synchronously wait
> in system thread for some message to come. When message handler gets to the
> same stripe Ignite hangs. I have a list of such places discovered so far.
> After I fix them I will try re-approach the pools.
> 
> Another reason for not switching to striped pool completely is that stripes
> are good for cache updates and single or batched get operations since they
> reduce inter-thread communication and synchronization to minimum, however I
> think that we should run, for example, SQL queries in old fashioned pools -
> first, they can fallback to scanning random data, second, we pause cache
> updates in some partition for query execution time. I was thinking of
> splitting CPU resources between striped pool and system pool in some ratio.
> I will follow up on this in a couple of days and will explain what has been
> done and what I want to do.
> 
> Vladimir, I hope I responded to your points.
> 
> Thanks!
> 
> --Yakov


Mime
View raw message