Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 9289B200D56 for ; Tue, 12 Dec 2017 09:18:43 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 90F08160C0F; Tue, 12 Dec 2017 08:18:43 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 6257A160BE7 for ; Tue, 12 Dec 2017 09:18:42 +0100 (CET) Received: (qmail 24316 invoked by uid 500); 12 Dec 2017 08:18:41 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 24305 invoked by uid 99); 12 Dec 2017 08:18:41 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Dec 2017 08:18:41 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id CD3941A088B for ; Tue, 12 Dec 2017 08:18:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.389 X-Spam-Level: ** X-Spam-Status: No, score=2.389 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, T_KAM_HTML_FONT_INVALID=0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=harri.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id PtxEUKgjMoIO for ; Tue, 12 Dec 2017 08:18:37 +0000 (UTC) Received: from mail-wm0-f66.google.com (mail-wm0-f66.google.com [74.125.82.66]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 533C35FB72 for ; Tue, 12 Dec 2017 08:18:37 +0000 (UTC) Received: by mail-wm0-f66.google.com with SMTP id b199so15283166wme.1 for ; Tue, 12 Dec 2017 00:18:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=harri.com; s=google; h=from:mime-version:subject:date:references:to:in-reply-to:message-id; bh=g2gFFhNEN1Jv952IrdelxPzaU++L1+Rw+N18LEJKSBE=; b=AnG3hnnAZuW0IROfCuXa0VDw6J3X2MGJB58dLRVmTo0JUDIQeoX9dGclAqHqEaqEIy ukzy31ZgzZIS8mGC0Bvx93LHpTzaZT8ywXEKkIs9nulgOjzCAEBn8ZlD2gCb9ns1E3zc kA2oLSSEDNaWFEa2gsFGCn2soa5H4xTpWFDqc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:date:references:to :in-reply-to:message-id; bh=g2gFFhNEN1Jv952IrdelxPzaU++L1+Rw+N18LEJKSBE=; b=o3+zWnO9ZFcb9l4L4yTOWnK1qFTLZkS5V8wYth7Vv+stzkN9MuJdGrhSOZp7JF164s yRAPPUSpVcXFafYzYv8KVR9zMkpNp85tdodFeN+XxUQpcEfiAeO0PYn8aRCklbiKj7Rd V4/zPU60MpxFl74zG9mrdQMe0ugVffeLusO8cXC980o+x0oznwc6m7G4UwGnQA9wwDhf K8TpQNXL5/qC4YHb/YTMGl+BwnUstE+tMaK0alSy6e8u4oRJdjw+Raflk89iQngzeLoc espu5IVOiJhawnZq9Pa8KAjRCI353v+6/zsM38RkfWBBV7WRfNCS/mT0ub//sD4GI3n4 N4wA== X-Gm-Message-State: AKGB3mK1CPyIjlKihBs47DmrKvePZ3Sd9vfIhGA0eruey+kFz+rjPrvJ Nwi8EaAhLb9FLFQ1KcIoDQ/Swd47vw0DYMyLywXuJkCoxpB0B78aM9qk0N+YoqeIp+RItbLrnQI wLM2SpdvBI4k4qQ== X-Google-Smtp-Source: ACJfBotcygRhKiL+3OFORsdHxmHwissIICPTwx/tXI9I3AQ1vLZIab1n65Q0d81dhMuqp7e3mEif5A== X-Received: by 10.28.247.15 with SMTP id v15mr869643wmh.152.1513066711550; Tue, 12 Dec 2017 00:18:31 -0800 (PST) Received: from [192.168.1.125] ([188.225.171.22]) by smtp.gmail.com with ESMTPSA id 192sm11713208wmg.32.2017.12.12.00.18.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Dec 2017 00:18:30 -0800 (PST) From: Ahmad Al-Masry Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Data lose in query Date: Tue, 12 Dec 2017 10:18:28 +0200 References: <48A3EC9A-7193-47CA-A19A-14A812312E71@harri.com> <7D64D283-BDD0-4CA0-B254-F8A90266F9DF@harri.com> To: user@ignite.apache.org In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3445.5.20) Content-Type: multipart/alternative; boundary="Apple-Mail=_4A6A0F95-7D74-4298-A826-0D540972717B" archived-at: Tue, 12 Dec 2017 08:18:43 -0000 --Apple-Mail=_4A6A0F95-7D74-4298-A826-0D540972717B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="ISO-8859-1" Hi; I added @AffinityKeyMapped to the fields in the model generated by web cons= ole. But I also noticed that when I load the data using the load commands genera= ted from the web console, the data is not collocated and the query will ret= urn reduced data. Do you have any hints? BR > On Dec 11, 2017, at 3:12 PM, Nikolai Tikhonov wrot= e: >=20 > It depends from your data model and can't be enabled via one property. Pl= ease, look at the following documentation pages: >=20 > https://apacheignite.readme.io/docs/affinity-collocation > https://apacheignite-sql.readme.io/docs/distributed-joins#collocated-join= s >=20 > On Mon, Dec 11, 2017 at 4:02 PM, Ahmad Al-Masry > wrote: > How can I enable this on the server configuration XML? > BR >=20 >=20 >> On Dec 11, 2017, at 2:31 PM, Nikolai Tikhonov > wrote: >>=20 >> Hi, >>=20 >> Strongly recommend to care about collocation of your data (as above sugg= ested by Vlad) instead of enable DistributedJoins flag. The performance of = this type of joins is worse then the performance of the affinity collocatio= n based joins due to the fact that there will be much more network round-tr= ips and data movement between the nodes to fulfill a query [1]. >>=20 >> 1. https://apacheignite-sql.readme.io/docs/distributed-joins#non-colloca= ted-joins >>=20 >>=20 >> On Mon, Dec 11, 2017 at 3:03 PM, Ahmad Al-Masry > wrote: >> Hi; >> When I enabled the distributed JOIN, get the following Exception: >>=20 >> java.sql.SQLException: javax.cache.CacheException: Failed to prepare dis= tributed join query: join condition does not use index [joinedCache=3DPosit= ionTypeCache >>=20 >> Should I remove the indexes before doing distributed joins? >> BR >>=20 >>=20 >>> On Dec 11, 2017, at 10:43 AM, Vladislav Pyatkov > wrote: >>>=20 >>> Hi, >>>=20 >>> When you use JOIN, you should to enable DistributedJoins flag[1], or ta= ck care about collocated of each joined entry[2]. >>>=20 >>> [1]: org.apache.ignite.cache.query.SqlFieldsQuery#setDistributedJoins >>> [2]: https://apacheignite.readme.io/docs >>>=20 >>> On Mon, Dec 11, 2017 at 11:36 AM, Ahmad Al-Masry > wrote: >>> Dears; >>> The when I execute the attached query on Mysql data source or on a sing= le node ignite, it returns about 25k records. >>> When multiple node, it gives me about 3500 records. >>> The caches are atomic and partitioned. >>> Any suggestions. >>> BR >>>=20 >>> -- >>>=20 >>>=20 >>>=20 >>> This email, and the content it contains, are intended only for the pers= ons >>> or entities to which it is addressed. It may contain sensitive, >>> confidential and/or privileged material. Any review, retransmission, >>> dissemination or other use of, or taking of any action in reliance upon= , >>> this information by persons or entities other than the intended >>> recipient(s) is prohibited. If you received this email in error, please >>> immediately contact security[at]harri[dot]com and delete it from any de= vice >>> or system on which it may be stored. >>>=20 >>>=20 >>>=20 >>> --=20 >>> Vladislav Pyatkov >>=20 >>=20 >>=20 >> This email, and the content it contains, are intended only for the perso= ns or entities to which it is addressed. It may contain sensitive, confiden= tial and/or privileged material. Any review, retransmission, dissemination = or other use of, or taking of any action in reliance upon, this information= by persons or entities other than the intended recipient(s) is prohibited.= If you received this email in error, please immediately contact security[a= t]harri[dot]com and delete it from any device or system on which it may be = stored. >>=20 >=20 >=20 >=20 > This email, and the content it contains, are intended only for the person= s or entities to which it is addressed. It may contain sensitive, confident= ial and/or privileged material. Any review, retransmission, dissemination o= r other use of, or taking of any action in reliance upon, this information = by persons or entities other than the intended recipient(s) is prohibited. = If you received this email in error, please immediately contact security[at= ]harri[dot]com and delete it from any device or system on which it may be s= tored. >=20 --=20 This email, and the content it contains, are intended only for the persons= =20 or entities to which it is addressed. It may contain sensitive,=20 confidential and/or privileged material. Any review, retransmission,=20 dissemination or other use of, or taking of any action in reliance upon,=20 this information by persons or entities other than the intended=20 recipient(s) is prohibited. If you received this email in error, please=20 immediately contact security[at]harri[dot]com and delete it from any device= =20 or system on which it may be stored. --Apple-Mail=_4A6A0F95-7D74-4298-A826-0D540972717B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="ISO-8859-1" Hi;
I ad= ded @AffinityKeyMapped to the fields in the model&nbs= p;generated by web console.
Bu= t I also noticed that when I load the data using the load commands gen= erated from the web console, the data is not collocated and the query = will return reduced data.
Do y= ou have any hints?
BR


On Dec 11, 2017, at 3:12 PM, Nik= olai Tikhonov <ntikho= nov@apache.org> wrote:

=
It depends from your data model= and can't be enabled via one property. Please, look at the following docum= entation pages:

https://apacheignite.readme.io/docs/affinity-collocation
https://apacheignite-s= ql.readme.io/docs/distributed-joins#collocated-joins

On Mon, Dec 11, 2017 at 4:02 PM, Ahmad Al-Masry <masry@harri.com> wrote:
How can I enable this on the server configuration XML?BR


O= n Dec 11, 2017, at 2:31 PM, Nikolai Tikhonov <ntikhonov@apache.org> wro= te:

Hi,

<= div class=3D"">Strongly recommend to care about collocation of your data (a= s above suggested by Vlad) instead of enable DistributedJoins flag. The per= formance of this type of joins is worse then the performance of the affinit= y collocation based joins due to the fact that there will be much more netw= ork round-trips and data movement between the nodes to fulfill a query [1].=

1. https://apacheig= nite-sql.readme.io/docs/distributed-joins#n= on-collocated-joins


On Mon, D= ec 11, 2017 at 3:03 PM, Ahmad Al-Masry <masry@harri.co= m> wrote:
Hi;When I enabled the distributed JOIN, get the following E= xception:

java.sq= l.SQLException: javax.cache.CacheException: Failed to prepare distributed j= oin query: join condition does not use index [joinedCache=3DPositionTypeCac= he

Should I remov= e the indexes before doing distributed joins?
BR


O= n Dec 11, 2017, at 10:43 AM, Vladislav Pyatkov <vldpyatkov@gmail.com> w= rote:

Hi,

When you use JOIN, you should t= o enable DistributedJoins flag[1], or tack care about collocated of ea= ch joined entry[2].

[1]: org.apache.ignite.cache.query.SqlFieldsQuery#setDis= tributedJoins
[2]: = ;https://apacheignite.readme.io/docs

On Mon, Dec= 11, 2017 at 11:36 AM, Ahmad Al-Masry <masry@harri.com= > wrote:
Dears;<= br class=3D""> The when I execute the attached query on Mysql data source or on a single n= ode ignite, it returns about 25k records.
When multiple node, it gives me about 3500 records.
The caches are atomic and partitioned.
Any suggestions.
BR

--



This email, and the content it contains, are intended only for the persons<= br class=3D""> or entities to which it is addressed. It may contain sensitive,
confidential and/or privileged material. Any review, retransmission,
dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended
recipient(s) is prohibited. If you received this email in error, please
immediately contact security[at]harri[dot]com and delete it from any device=
or system on which it may be stored.



--
Vladislav Pyatkov



This email, and the content it contains, are intended only for th= e persons or entities to which it is addressed. It may contain sensitive, c= onfidential and/or privileged material. Any review, retransmission, dissemi= nation or other use of, or taking of any action in reliance upon, this info= rmation by persons or entities other than the intended recipient(s) is proh= ibited. If you received this email in error, please immediately contact sec= urity[at]harri[dot]com and delete it from any device or system on which it = may be stored.




This email, and the content it contains, are intended o= nly for the persons or entities to which it is addressed. It may contain se= nsitive, confidential and/or privileged material. Any review, retransmissio= n, dissemination or other use of, or taking of any action in reliance upon,= this information by persons or entities other than the intended recipient(= s) is prohibited. If you received this email in error, please immediately c= ontact security[at]harri[dot]com and delete it from any device or system on= which it may be stored.
=



This email, and the content it c= ontains, are intended only for the persons or entities to which it is addre= ssed. It may contain sensitive, confidential and/or privileged material. An= y review, retransmission, dissemination or other use of, or taking of any a= ction in reliance upon, this information by persons or entities other than = the intended recipient(s) is prohibited. If you received this email in erro= r, please immediately contact security[at]harri[dot]com and delete it from = any device or system on which it may be stored.

 --Apple-Mail=_4A6A0F95-7D74-4298-A826-0D540972717B--