Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id AA4D8200BA6 for ; Tue, 18 Oct 2016 09:06:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A8ECF160ADC; Tue, 18 Oct 2016 07:06:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CA122160ACC for ; Tue, 18 Oct 2016 09:06:21 +0200 (CEST) Received: (qmail 20705 invoked by uid 500); 18 Oct 2016 07:06:20 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 20696 invoked by uid 99); 18 Oct 2016 07:06:20 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Oct 2016 07:06:20 +0000 Received: from mail-lf0-f52.google.com (mail-lf0-f52.google.com [209.85.215.52]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 4BB151A00D6 for ; Tue, 18 Oct 2016 07:06:20 +0000 (UTC) Received: by mail-lf0-f52.google.com with SMTP id l131so12284189lfl.2 for ; Tue, 18 Oct 2016 00:06:20 -0700 (PDT) X-Gm-Message-State: AA6/9RkqVNEbaU7AP6dyfh3dgQjT282jmuZUAHnhwRlRPkSd+OtNfWDua3zHGBJ2q0PemfPjQVjKpGTvAecWyxO2 X-Received: by 10.25.16.39 with SMTP id f39mr18617870lfi.166.1476774377814; Tue, 18 Oct 2016 00:06:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.159.17 with HTTP; Tue, 18 Oct 2016 00:06:16 -0700 (PDT) In-Reply-To: <2016101814345612246379@neusoft.com> References: <2016101814345612246379@neusoft.com> From: Alexey Kuznetsov Date: Tue, 18 Oct 2016 14:06:16 +0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Can't increase the speed of loadCache() when increasing more Ignite node To: user@ignite.apache.org Content-Type: multipart/alternative; boundary=001a113fc3623414e5053f1e54d9 archived-at: Tue, 18 Oct 2016 07:06:22 -0000 --001a113fc3623414e5053f1e54d9 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Bob, In current Ignite implementation, if you are loading data via cache store each node will iterate whole data set and take only those keys that will satisfy affinity function. So, in case of one node - all keys will loaded. in case of two nodes: first node will iterate WHOLE data set, but will take 50% of keys, same with second node. in case of three nodes: each node will iterate WHOLE data set, but will take 33% of keys. There is no general solution to speedup load. On Tue, Oct 18, 2016 at 1:34 PM, =E8=83=A1=E6=B0=B8=E4=BA=AE/Bob wrote: > Hi, > > I load data into Ignite from oracle with loadCache(). > > I load 100w data, when Ignite cluster has one node, its cost time is > 2m27s. > Two nodes, its cost time is 2m18s, three 2m15s, four 2m11s. > > I have tested reading the 100w data through jdbc, its cost time is 40= s. > > Why don't the loading speed increase with mode Ignite nodes? > > Thanks. > > ------------------------------ > Bob > > ------------------------------------------------------------ > --------------------------------------- > Confidentiality Notice: The information contained in this e-mail and any > accompanying attachment(s) > is intended only for the use of the intended recipient and may be > confidential and/or privileged of > Neusoft Corporation, its subsidiaries and/or its affiliates. If any reade= r > of this communication is > not the intended recipient, unauthorized use, forwarding, printing, > storing, disclosure or copying > is strictly prohibited, and may be unlawful.If you have received this > communication in error,please > immediately notify the sender by return e-mail, and delete the original > message and all copies from > your system. Thank you. > ------------------------------------------------------------ > --------------------------------------- > --=20 Alexey Kuznetsov --001a113fc3623414e5053f1e54d9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Bob,

In current Ignite implementation, = if you are loading data via cache store each node will iterate whole data s= et
=C2=A0and take only those keys that will satisfy affinity func= tion.

So,
=C2=A0in case of one node - al= l keys will loaded.
=C2=A0in case of two nodes: first node will i= terate WHOLE data set, but will take 50% of keys, same with second node.
=C2=A0in case of three nodes: each node will iterate WHOLE data= set, but will take 33% of keys.

There is no gener= al solution to speedup load.


On Tue, Oct 18, 2016 at 1:34 PM, =E8= =83=A1=E6=B0=B8=E4=BA=AE/Bob <hu-yl@neusoft.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
Hi,=C2=A0

=C2=A0 =C2=A0 I load data into Ig= nite from oracle with loadCache().

=C2=A0 =C2=A0 I load 1= 00w data, when Ignite cluster has one node, its cost time is 2m27s.<= /div>
= =C2=A0 =C2=A0 Two nodes, its cost time is=C2=A02m18s, three=C2= =A02m15s, four=C2=A02m11s.
=C2=A0 =C2=A0 I have tested reading the 100w data through jdbc, it= s cost time is 40s.

=C2=A0 =C2=A0 Why don&= #39;t the loading speed increase with mode Ignite nodes?
<= span style=3D"color:rgb(0,0,0);background-color:rgba(0,0,0,0)">
<= /div>
= =C2=A0 =C2=A0 Thanks.

Bo= b

--------------------------------------------------------------= -------------------------------------
Confidentiality Notice: The i= nformation contained in this e-mail and any accompanying attachment(s)
= is intended only for the use of the intended recipient and may be confident= ial and/or privileged of
Neusoft Corporation, its subsidiaries and/or i= ts affiliates. If any reader of this communication is
not the intended = recipient, unauthorized use, forwarding, printing,=C2=A0 storing, disclosur= e or copying
is strictly prohibited, and may be unlawful.If you have re= ceived this communication in error,please
immediately notify the sender= by return e-mail, and delete the original message and all copies from
= your system. Thank you.
------------------------------------------= ---------------------------------------------------------




--
Alexey Kuznetsov
--001a113fc3623414e5053f1e54d9--