ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Di Tommaso <paolo.ditomm...@gmail.com>
Subject Re: Is it safe to modify the activeJobs and waitingJobs collections in the CollisionContext?
Date Mon, 01 Feb 2016 11:24:48 GMT
Not sure that load balancing is what is needed in this case.

The problem is to make sure that at any time tasks do not consume more
resources than the available ones. Thus, a collision spi is the right
strategy to handle that

I've ended up implementing a custom collision spi delegating the job
stealing to the default implementation. It looks working fine. You can find
the code here:

https://github.com/nextflow-io/nextflow/blob/master/subprojects/nxf-ignite/src/main/nextflow/collision/CustomStealingCollisionSpi.groovy#L60



Best,
Paolo


On Fri, Jan 29, 2016 at 11:26 PM, vkulichenko <valentin.kulichenko@gmail.com
> wrote:

> Paolo,
>
> This actually sounds like a use case for load balancing [1]. In particular,
> I think you can use AdaptiveLoadBalancingSpi for what you described, but
> you
> will have to provide your own implementation of AdaptiveLoadProbe, because
> we don't currently have the one based on memory utilization.
>
> Will this work for you?
>
> [1] https://apacheignite.readme.io/docs/load-balancing
>
> -Val
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Is-it-safe-to-modify-the-activeJobs-and-waitingJobs-collections-in-the-CollisionContext-tp2758p2779.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Mime
View raw message