From issues-return-96122-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Mon May 20 08:56:07 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 715F2180627 for ; Mon, 20 May 2019 10:56:07 +0200 (CEST) Received: (qmail 21045 invoked by uid 500); 20 May 2019 08:56:03 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 20955 invoked by uid 99); 20 May 2019 08:56:03 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 May 2019 08:56:03 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B5969E2C52 for ; Mon, 20 May 2019 08:56:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 132C325822 for ; Mon, 20 May 2019 08:56:01 +0000 (UTC) Date: Mon, 20 May 2019 08:56:01 +0000 (UTC) From: "Alexey Goncharuk (JIRA)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (IGNITE-10619) Add support files transmission between nodes over connection via CommunicationSpi MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/IGNITE-10619?page=3Dcom.atlassi= an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16= 843788#comment-16843788 ]=20 Alexey Goncharuk commented on IGNITE-10619: ------------------------------------------- [~Mmuzaf], folks, The current API looks too complex to me. Are there any reasons to introduce= our own Channel, ChannelId, IgniteSocketChannel, etc... ? Ignite is not a = framework to transfer files, so internal API should be kept as simple as po= ssible. I would suggest the following change to the GridIOManager: {code} public IgniteFuture openChannel(ClusterNode rmtN= ode, Message channelInitMessage); public void addChannelListener(ChannelListener lsnr); interface ChannelListener { onChannelOpened(ClusterNode rmtNode, Message initMessage, Channel chann= el); }=20 {code} The IgniteFileTransmitProcessor API also feels too redundant. The only thin= g that needs to be provided on the supplier side is an input stream/channel= . I see no need in {{TransmitSessionFactory}} and sibling classes. [~DmitriyGovorukhin], [~ibessonov], can you take a look at the change as we= ll? > Add support files transmission between nodes over connection via Communic= ationSpi > -------------------------------------------------------------------------= -------- > > Key: IGNITE-10619 > URL: https://issues.apache.org/jira/browse/IGNITE-10619 > Project: Ignite > Issue Type: Sub-task > Components: persistence > Reporter: Maxim Muzafarov > Assignee: Maxim Muzafarov > Priority: Major > Labels: iep-28 > > Partition preloader must support cache partition file relocation from one= cluster node to another (the zero copy algorithm [1] assume to be used by = default). To achieve this, the file transfer machinery must be implemented = at Apache Ignite over Communication SPI. > _CommunicationSpi_ > Ignite's Comminication SPI must support to: > * establishing channel connections to the remote node to an arbitrary top= ic (GridTopic) with predefined processing policy; > * listening incoming channel creation events and registering connection h= andlers on the particular node; > * an arbitrary set of channel parameters on connection handshake; > _FileTransmitProcessor_ > The file transmission manager must support to: > * using different approaches of incoming data handling =E2=80=93 buffered= and direct (zero-copy approach of FileChannel#transferTo); > * transferring data by chunks of predefined size with saving intermediate= results; > * re-establishing connection if an error occurs and continue file upload\= download; > * limiting connection bandwidth (upload and download) at runtime; -- This message was sent by Atlassian JIRA (v7.6.3#76005)