ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Goncharuk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-10619) Add support files transmission between nodes over connection via CommunicationSpi
Date Mon, 20 May 2019 08:56:01 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843788#comment-16843788
] 

Alexey Goncharuk commented on IGNITE-10619:
-------------------------------------------

[~Mmuzaf], folks,

The current API looks too complex to me. Are there any reasons to introduce our own Channel,
ChannelId, IgniteSocketChannel, etc... ? Ignite is not a framework to transfer files, so internal
API should be kept as simple as possible.

I would suggest the following change to the GridIOManager:
{code}
public IgniteFuture<java.nio.channels.Channel> openChannel(ClusterNode rmtNode, Message
channelInitMessage);

public void addChannelListener(ChannelListener lsnr);

interface ChannelListener {
    onChannelOpened(ClusterNode rmtNode, Message initMessage, Channel channel);
} 
{code}

The IgniteFileTransmitProcessor API also feels too redundant. The only thing that needs to
be provided on the supplier side is an input stream/channel. I see no need in {{TransmitSessionFactory}}
and sibling classes.

[~DmitriyGovorukhin], [~ibessonov], can you take a look at the change as well?

> Add support files transmission between nodes over connection via CommunicationSpi
> ---------------------------------------------------------------------------------
>
>                 Key: IGNITE-10619
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10619
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: persistence
>            Reporter: Maxim Muzafarov
>            Assignee: Maxim Muzafarov
>            Priority: Major
>              Labels: iep-28
>
> Partition preloader must support cache partition file relocation from one cluster node
to another (the zero copy algorithm [1] assume to be used by default). To achieve this, the
file transfer machinery must be implemented at Apache Ignite over Communication SPI.
> _CommunicationSpi_
> Ignite's Comminication SPI must support to:
> * establishing channel connections to the remote node to an arbitrary topic (GridTopic)
with predefined processing policy;
> * listening incoming channel creation events and registering connection handlers on the
particular node;
> * an arbitrary set of channel parameters on connection handshake;
> _FileTransmitProcessor_
> The file transmission manager must support to:
> * using different approaches of incoming data handling – buffered and direct (zero-copy
approach of FileChannel#transferTo);
> * transferring data by chunks of predefined size with saving intermediate results;
> * re-establishing connection if an error occurs and continue file upload\download;
> * limiting connection bandwidth (upload and download) at runtime;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message