ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: [DISCUSSION] Channel communication between nodes
Date Tue, 05 Mar 2019 14:52:45 GMT
Maxim,

My humble opinion. If there is no convenient means to implement
partition file sending today then we should introduce something. And
keeping such facility private is much easier, because introduction of
new public API is a significantly more complex task.

пт, 1 мар. 2019 г. в 19:44, Maxim Muzafarov <maxmuzaf@gmail.com>:
>
> Igniters,
>
> Apache Ignite has a very suitable messaging user interface [1] for
> topic-based communication between nodes (or a specific group of nodes
> within a cluster). The messaging functionality in Ignite is provided
> via IgniteMessaging interface. It allows:
> - send a message to a certain topic
> - register local\remote listeners
>
> I really like this feature, but the disadvantage here is when the user
> wants to transfer a large amount of binary data (e.g. files) between
> nodes he must create a complex logic to wrap it into messages. I think
> Ignite could have an interface e.g. IgniteChannels which will allow:
> - register local\remote listeners for channel created\destroy events.
> - create a channel connection (a wrapped socket channel) to a certain
> node\group of nodes and the desired topic
>
> As another suitable case where such a feature can be applied is
> internal usage for Apache Ignite needs. I can mention here the task of
> cluster rebalancing by sending cache partition files between nodes.
> I've posted a small description of it on the IEP-28 page [2].
>
>
> WDYT about it?
>
> ---
>
> API (assumed)
>
> IgniteChannels chnls = ignite0.channels();
> chnls.remoteListen(TOPIC.MY_TOPIC, new RemoteListener());
>
> IgniteSocketChannel ch0 = chnls.channel(node, TOPIC.MY_TOPIC);
> ch0.writeInt(bigFile.size());
> ch0.transferTo(FileChannel.open(bigFile.path(), StandardOpenOption.READ))
>
>
> /** */
>
> private class RemoteListener
>     implements IgniteBiPredicate<UUID, IgniteSocketChannel> {
>
>     @IgniteInstanceResource
>     private Ignite ignite;
>
>     @Override public boolean apply(
>         UUID nodeId,
>         IgniteSocketChannel ch
>     ) {
>         int size = ch.readInt();
>         ignite.fileSystem("base")
>             .create("bigfile.mpg")
>             .transferFrom(ch, size);
>         return true;
>     }
> }
>
>
> [1] https://apacheignite.readme.io/docs/messaging
> [2] https://cwiki.apache.org/confluence/display/IGNITE/IEP-28%3A+Cluster+peer-2-peer+balancing#IEP-28:Clusterpeer-2-peerbalancing-CommunicationSpi



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message