ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakov Zhdanov <yzhda...@apache.org>
Subject Re: Computation on NodeEntries
Date Fri, 11 Dec 2015 13:06:16 GMT
Guys, I would do the following:

1. Map all my partitions to
nodes: org.apache.ignite.cache.affinity.Affinity#mapPartitionsToNodes
2. Send jobs (with its list of partitions) to each node using map returned
on step1
3. Job may be like:

new Runnable() {
    @Override public void run() {
        for (Integer part : parts) {
            Iterator<Cache.Entry<Object, Object>> it = cache.query(new
ScanQuery<>(part)).iterator();

            // do the stuff...
        }

    }
};

This may result in network calls for some worst cases when topology
changes under your feet, but even in this case this should work.


--Yakov

2015-12-11 2:13 GMT+03:00 Andrey Kornev <andrewkornev@hotmail.com>:

> Dmitriy,
>
> Given the approach you suggested below, what would be your recommendation
> for dealing with cluster topology changes while the iteration is in
> progress? An obvious one I can think of is to
> - somehow detect the change,
> - cancel the tasks on all the nodes
> - wait until the rebalancing is finished and
> - restart the computation.
>
> Are there any other ways? Ideally, I'd like to have the "exactly-once"
> execution semantics.
>
> Thanks
> Andrey
>

Mime
View raw message