Return-Path: X-Original-To: apmail-ignite-user-archive@minotaur.apache.org Delivered-To: apmail-ignite-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8710119367 for ; Tue, 5 Apr 2016 09:08:59 +0000 (UTC) Received: (qmail 14808 invoked by uid 500); 5 Apr 2016 09:08:59 -0000 Delivered-To: apmail-ignite-user-archive@ignite.apache.org Received: (qmail 14759 invoked by uid 500); 5 Apr 2016 09:08:59 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 14749 invoked by uid 99); 5 Apr 2016 09:08:59 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Apr 2016 09:08:59 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id E6790180534 for ; Tue, 5 Apr 2016 09:08:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.279 X-Spam-Level: * X-Spam-Status: No, score=1.279 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gridgain-com.20150623.gappssmtp.com Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id T7lIijyzqWYk for ; Tue, 5 Apr 2016 09:08:56 +0000 (UTC) Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com [209.85.217.180]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTPS id 3F2015F471 for ; Tue, 5 Apr 2016 09:08:56 +0000 (UTC) Received: by mail-lb0-f180.google.com with SMTP id qe11so5064363lbc.3 for ; Tue, 05 Apr 2016 02:08:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gridgain-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=b13LE1yzO1C6hKz0G/O7j47J/MMqD9qYjbFIlO7Rm1Y=; b=k17+t/LdSj4yql3lmgLQkAmMV7/0xoORJmpzqRojMyeFWzO1I3XtnO6vj2es95mG53 s4ULfSbv3cIPutY63d5x6+befYO9QHp7lZcD2iQAA4pm7LXeSeKqe/4Z01vTA02VHDCJ YulB9TsnI/D2Q9xiHsdFGtcbHSG64/TWvf2LNLdbDiZzISsu2hlLl6lUw8y2UZIeM6kc ulRG2W8+Dy3I21TcbRbgHuRQq4/v/jp0vjnOPED+2Iy35wi+2vv9rpGrt/ffInyhkhXn 8CsoUBHQoFAD9YzBdDIA+5svIjhwkU04l794u+BvkXW9Tn0/Kf+3LsZrLEr6k3OqtK8m JsYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=b13LE1yzO1C6hKz0G/O7j47J/MMqD9qYjbFIlO7Rm1Y=; b=GbevVRgNDjDT6xF7anQMhspvGwVPzuyTjkK+yIzVOmxhoQwWW4T7k5fIdHc3N8Gt84 4LSLLankd6Wu1zcgEYWZrh9ZDLWBtBnGu4au9rVMcEQ3ozW3cSe6ulmsXxWVpPOe19BW EMkSDg4J7yyQcQYVWQe8SFVLgh+vJnl5Wi/Dw4KNvFwIl04x2hVRnecUruAm+s3HAaK6 N3wXKM8ST/PGV9SGoyHRmkHXVroVnOYYyqDw3gLg7YdwlsYczJ/VxiWkFqPxMRtpcBBh yZNTGrEcPrOV5W7YQPDSc0BD7E6P/YuUr/wu/pdCJA7koWXpIt4LH973pn37adPyi5MS BW4w== X-Gm-Message-State: AD7BkJI+wqDK07m+mOrsIlSOx6bd8vW4nVryRg4kKYRs4VXwBr0TLjr83lGtNMStE19kI1nd X-Received: by 10.112.130.41 with SMTP id ob9mr6560671lbb.81.1459847335717; Tue, 05 Apr 2016 02:08:55 -0700 (PDT) Received: from [10.0.0.18] ([109.188.127.5]) by smtp.googlemail.com with ESMTPSA id i204sm4202058lfg.47.2016.04.05.02.08.54 for (version=TLSv1/SSLv3 cipher=OTHER); Tue, 05 Apr 2016 02:08:54 -0700 (PDT) Subject: Re: Running a query in only current nodes partitions To: user@ignite.apache.org References: <1459769388424-3890.post@n6.nabble.com> From: Denis Magda Message-ID: <570380AB.3010507@gridgain.com> Date: Tue, 5 Apr 2016 12:08:59 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------040400040705070507010401" This is a multi-part message in MIME format. --------------040400040705070507010401 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Hi Tolga, Probably you can use SqlQuery as is by getting a list of keys that have to be updated with"SELECT _key FROM cache2 WHERE ..." where "_key" is Ignite specific keyword saying that entries key has to be included into the result set. However since SqlQueries are broadcasted to every node (if cache mode is PARTITIONED) the following has to be considered (can be ignored for REPLICATED caches): - indexes has to be properly configured for "cache2". Execution plan can be checked with "EXPLAIN SELECT ..." - frequency of such queries. If the query will be executed too frequent it may affect performance. - size of the result set. If you return tens or hundreds of keys in a result set then it's ok but if the size is measured in thousands and thousands of rows it may have a negative impact. If solution with SqlQuery works fine then you can iterate over the keys local preparing updates for cache1 and then use cache1.putAll to apply changes. Will this work for you? -- Denis On 4/4/2016 3:11 PM, Tolga Kavukcu wrote: > Hi Denis, > > Thanks for the answer. > > Better if i provide more detail to get a point . Lets say i have > cache1 and cache2. > > - I would like to run a query on cache1. Than i should check if key > exists or not in cache2 than i will execute some logic. > > I need to make sure that keys are owned by queried node so that i > could check properly if key exists in cache or not .(This can be > achieved by ScanQuery by setting partition and setLocal=true) > > So if i use sql query my query would be like ; > > SqlQuery sql = new SqlQuery(Person.class,"salary > ?"); > > I only apply one rule over one field. > > But! there is no setPartition() method in SqlQuery. Please correct me > if am wrong. > > Thats why i use ScanQuery to itearte over cache and apply my rule. > > I will try multi-thread approach, it could speed up think. Also i > would be happy if you can suggest a faster alternative way. > > Thanks. > > On Mon, Apr 4, 2016 at 2:29 PM, Denis Magda > wrote: > > Hi Tolga, > > Do you really need to iterate over *all the keys* when data has to be > update? If so then you can parallelize ScanQueries when multiple local > Threads will be iterating over specific partitions on each node. > Please refer to this example for more details > https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java > > If you don't need to iterate over every key then how would your > query look > like if you use SQL query to get a subset of the keys? > > -- > Denis > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Running-a-query-in-only-current-nodes-partitions-tp3878p3890.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > > > > > -- > *Tolga KAVUKÇU > * --------------040400040705070507010401 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Hi Tolga,

Probably you can use SqlQuery as is by getting a list of keys that have to be updated with"SELECT _key FROM cache2 WHERE ..." where "_key" is Ignite specific keyword saying that entries key has to be included into the result set.
However since SqlQueries are broadcasted to every node (if cache mode is PARTITIONED) the following has to be considered (can be ignored for REPLICATED caches):
- indexes has to be properly configured for "cache2". Execution plan can be checked with "EXPLAIN SELECT ..."
- frequency of such queries. If the query will be executed too frequent it may affect performance.
- size of the result set. If you return tens or hundreds of keys in a result set then it's ok but if the size is measured in thousands and thousands of rows it may have a negative impact.

If solution with SqlQuery works fine then you can iterate over the keys local preparing updates for cache1 and then use cache1.putAll to apply changes.

Will this work for you?

--
Denis


On 4/4/2016 3:11 PM, Tolga Kavukcu wrote:
Hi Denis,

Thanks for the answer.

Better if i provide more detail to get a point . Lets say i have cache1 and cache2. 

- I would like to run a query on cache1. Than i should check if key exists or not in cache2 than i will execute some logic.

I need to make sure that keys are owned by queried node so that i could check properly if key exists in cache or not .(This can be achieved by ScanQuery by setting partition and setLocal=true)

So if i use sql query my query would be like ;

SqlQuery sql = new SqlQuery(Person.class, "salary > ?");

I only apply one rule over one field.

But! there is no setPartition() method in SqlQuery. Please correct me if am wrong.

Thats why i use ScanQuery to itearte over cache and apply my rule. 

I will try multi-thread approach, it could speed up think. Also i would be happy if you can suggest a faster alternative way.

Thanks.

On Mon, Apr 4, 2016 at 2:29 PM, Denis Magda <dmagda@gridgain.com> wrote:
Hi Tolga,

Do you really need to iterate over *all the keys* when data has to be
update? If so then you can parallelize ScanQueries when multiple local
Threads will be iterating over specific partitions on each node.
Please refer to this example for more details
https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java

If you don't need to iterate over every key then how would your query look
like if you use SQL query to get a subset of the keys?

--
Denis



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Running-a-query-in-only-current-nodes-partitions-tp3878p3890.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



--
Tolga KAVUKÇU


--------------040400040705070507010401--