Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BE604200D48 for ; Wed, 29 Nov 2017 20:14:54 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id BD24F160C04; Wed, 29 Nov 2017 19:14:54 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1015A160C02 for ; Wed, 29 Nov 2017 20:14:53 +0100 (CET) Received: (qmail 62217 invoked by uid 500); 29 Nov 2017 19:14:53 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 62207 invoked by uid 99); 29 Nov 2017 19:14:53 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Nov 2017 19:14:52 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 200171A0174 for ; Wed, 29 Nov 2017 19:14:52 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.222 X-Spam-Level: *** X-Spam-Status: No, score=3.222 tagged_above=-999 required=6.31 tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_ENVFROM_END_DIGIT=0.25, KAM_ASCII_DIVIDERS=0.8, NML_ADSP_CUSTOM_MED=1.2, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.972] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id uU-x0kA0RzwI for ; Wed, 29 Nov 2017 19:14:51 +0000 (UTC) Received: from n6.nabble.com (n6.nabble.com [162.255.23.37]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id EDC005F1B3 for ; Wed, 29 Nov 2017 19:14:50 +0000 (UTC) Received: from n6.nabble.com (localhost [127.0.0.1]) by n6.nabble.com (Postfix) with ESMTP id AAFED32A0199 for ; Wed, 29 Nov 2017 12:14:50 -0700 (MST) Date: Wed, 29 Nov 2017 12:14:50 -0700 (MST) From: soroka21 To: user@ignite.apache.org Message-ID: <1511982890697-0.post@n6.nabble.com> In-Reply-To: <1511883418595-0.post@n6.nabble.com> References: <1511811601304-0.post@n6.nabble.com> <1511829852960-0.post@n6.nabble.com> <1511883418595-0.post@n6.nabble.com> Subject: Re: SQL Count(*) returns incorrect count MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit archived-at: Wed, 29 Nov 2017 19:14:54 -0000 Switched to Ignite 2.3.0 in hope it has better behavior. Unfortunately it is not. During the execution of Spark job number of cache rows is growing but after Spark job completes - looks like some entries has been removed. JavaIgniteRDD shows correct count but again final result is incorrect. I was using sqlline to track count of record during Spark job execution - it shows some problem also: 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 6428 | +--------------------------------+ 1 row selected (0.063 seconds) 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 62501 | +--------------------------------+ 1 row selected (0.096 seconds) 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 268183 | +--------------------------------+ 1 row selected (0.154 seconds) 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 482616 | +--------------------------------+ 1 row selected (0.159 seconds) 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 722436 | +--------------------------------+ 1 row selected (0.159 seconds) 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 1017236 | <--------------------------- Still looks good here +--------------------------------+ 1 row selected (0.205 seconds) 0: jdbc:ignite:thin://10.238.42.86/> select count(*) from data3; +--------------------------------+ | COUNT(*) | +--------------------------------+ | 542535 | <--------------------------- Ooops! +--------------------------------+ -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/