Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 090D5200D50 for ; Mon, 4 Dec 2017 15:00:24 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 07894160C05; Mon, 4 Dec 2017 14:00:24 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 7A94C160BF9 for ; Mon, 4 Dec 2017 15:00:21 +0100 (CET) Received: (qmail 65479 invoked by uid 500); 4 Dec 2017 14:00:20 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 65469 invoked by uid 99); 4 Dec 2017 14:00:20 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Dec 2017 14:00:20 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 699E4C4D9C for ; Mon, 4 Dec 2017 14:00:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.845 X-Spam-Level: * X-Spam-Status: No, score=1.845 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.053, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=radcom.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id R5-17-AO9rsf for ; Mon, 4 Dec 2017 14:00:13 +0000 (UTC) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0139.outbound.protection.outlook.com [104.47.1.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 66FD260E3F for ; Mon, 4 Dec 2017 14:00:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=radcom.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3pYGQUicSWDMVY944oWKXBQpVvtKCeO1TxBNa4cz4jg=; b=LdVRnOIc1tVXfQRTg+x1j8ZbHoCrh5KaVqAe0T1puFMWZBnuwf31KPzALzYcMOBfwnPjMk1lqW7dkt8nuvW6a9YXQk0HSJ8XvHkrV+CP8M+KRpLDdm6sNEvHEEAK+/x0CJpgqADwyERG/QUNlYmRGV2GnWCNhWW2g79bPPLVeWc= Received: from HE1PR05MB0972.eurprd05.prod.outlook.com (10.161.115.11) by HE1PR05MB0969.eurprd05.prod.outlook.com (10.161.115.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.282.5; Mon, 4 Dec 2017 14:00:04 +0000 Received: from HE1PR05MB0972.eurprd05.prod.outlook.com ([fe80::6c75:cd65:4a:336c]) by HE1PR05MB0972.eurprd05.prod.outlook.com ([fe80::6c75:cd65:4a:336c%16]) with mapi id 15.20.0282.010; Mon, 4 Dec 2017 14:00:04 +0000 From: Sasha Haykin To: "user@ignite.apache.org" Subject: Ignite poor performance Thread-Topic: Ignite poor performance Thread-Index: AdNtAboREobHFfBYQwO+6GOpgUCwNA== Date: Mon, 4 Dec 2017 14:00:04 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=SashaH@radcom.com; x-originating-ip: [80.74.109.2] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;HE1PR05MB0969;6:4N1sqUz0uhwyq4o6uaYu1xqX0HdMqZ0wDNsPfzmhnOfT4OjuShAfaA+VzH2U+pxFCCHmKNLDoGz0tJcKmRXxehxLYqv6/5LMZavinBd3eqaAOMFEEPZVCreg9y1ZqO7A7B/jiZM6v7nQ7MGxbALge5jQcjAoX4bmAkkuPq8+mlyeL81Kl7w4qBRbJVSFJZX4NZcBycqhs0r8A+BeGAG67Xyl4WrBvDkqBWTJ1jIUPmC8p0PUI5s3Y4HzdtBuMy6Oxo1j3Af7eZhs/Suidm2EfD5wIN1zekjSO0gDQTKUEkW0zOIm7r+NV5s1t4TIVMqaX5va/nQ42OvLrMh778IgSqzUBQJgyGF1qEaKoD6RYNc=;5:JVfLapsNqqeRsvi+QdF5GG2D6OBsOJiOorsc7VxRAj5idKjqTBDIlkGG8p/gyYI0z0n+cHBVNK5cmCLP8p3P6KsA+jsiDep4nj2UHTh++da03WCY88p2s1nWKqml94R5SyPXGmoJ5VDbksMvFuWdme9DaIWycTGxw+zgL2GPG1M=;24:G8Ue/uJt4xGuGLYpIgPX8JYSbsBSA2KK8Ea0rc83AHtPSmDTJxSAgdA5pNbScD793eYoFlcUglhA7HfTwUe+M50JNk44QD7xzPveCFMklK8=;7:mO4G+up33rFq2EGwaSniViBP8tmfHyWZyo0TrwdWkSemFpcDbQ7mIJIMD/bYBPNo2Zn5rN3nf3MX4EomxpsKsVxCDJJWPfDrqOtKXOnFva+VaDY803/XnxQBytP4YIc6mipLXQhaCcFH741l9LLn8PT3ddSDvPBfR0EokSy9Lzy++9/0QpO9r7GHIj94F1x8/wcwCulkS5hp+N3xzPcZG1eFbL+Xz6wI3OotSHZiSxLh0ZTDAP8OPwAzhBQZY5sT x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 0a3c926e-4d59-43bd-30e0-08d53b1f520b x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603286)(49563074);SRVR:HE1PR05MB0969; x-ms-traffictypediagnostic: HE1PR05MB0969: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(250069074691196)(158342451672863)(46954378381280)(227612066756510)(21748063052155); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(102415395)(6040450)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3002001)(3231022)(6041248)(20161123562025)(20161123564025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123558100)(6072148)(201708071742011);SRVR:HE1PR05MB0969;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:HE1PR05MB0969; x-forefront-prvs: 051158ECBB x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(39830400002)(376002)(346002)(189002)(377424004)(288314003)(199003)(52164004)(6436002)(86362001)(5660300001)(2900100001)(99936001)(6916009)(966005)(733005)(6506006)(316002)(478600001)(7116003)(790700001)(6116002)(102836003)(3846002)(25786009)(3660700001)(72206003)(7736002)(561924002)(68736007)(3280700002)(74316002)(81156014)(81166006)(8676002)(1730700003)(189998001)(106356001)(2351001)(861006)(8936002)(101416001)(2906002)(606006)(66066001)(105586002)(9326002)(3480700004)(236005)(54896002)(9686003)(5640700003)(54356011)(53946003)(55016002)(6306002)(97736004)(14454004)(99286004)(80792005)(7696005)(5250100002)(54556002)(53936002)(2501003)(33656002);DIR:OUT;SFP:1102;SCL:1;SRVR:HE1PR05MB0969;H:HE1PR05MB0972.eurprd05.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; received-spf: None (protection.outlook.com: radcom.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/related; boundary="_004_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_"; type="multipart/alternative" MIME-Version: 1.0 X-OriginatorOrg: radcom.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a3c926e-4d59-43bd-30e0-08d53b1f520b X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Dec 2017 14:00:04.8502 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0eb9e2d9-8763-412e-9709-3f539e9e25bc X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR05MB0969 archived-at: Mon, 04 Dec 2017 14:00:24 -0000 --_004_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_ Content-Type: multipart/alternative; boundary="_000_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_" --_000_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, I Working on big POC for telecom industry solution I have loaded to ignite 5,000,000 objects/rows (I loaded it in the client s= ide Please find the code below) In the metric the used of heap looks too small (for 5M rows) is it normal u= tilization? [04-12-2017 13:10:27][INFO ][grid-timeout-worker-#23][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=3Dc4f42f84, uptime=3D00:37:00.639] ^-- H/N/C [hosts=3D2, nodes=3D2, CPUs=3D12] ^-- CPU [cur=3D0.13%, avg=3D4.76%, GC=3D0%] ^-- PageMemory [pages=3D818037] ^-- Heap [used=3D1655MB, free=3D59.53%, comm=3D4090MB] ^-- Non heap [used=3D63MB, free=3D95.82%, comm=3D65MB] ^-- Public thread pool [active=3D0, idle=3D1, qSize=3D0] ^-- System thread pool [active=3D0, idle=3D8, qSize=3D0] ^-- Outbound messages queue [size=3D0] [04-12-2017 13:10:27][INFO ][grid-timeout-worker-#23][IgniteKernal] FreeLis= t [name=3Dnull, buckets=3D256, dataPages=3D600522, reusePages=3D0] When I'm querying the data VIA JDBC the performance is very bad (I'm compar= ing it to VERTICA DB) What I'm doing wrong? The success factors for the POC is to query from 100M rows of data (with = filters on dates+ filter on one element ) and get results in 3 seconds is i= t possible to achieve it with Ignite? Here are the example of the data + query: SELECT count(*) FROM HPA WHERE DEST_NE_NAME=3D'destDest17' ; COUNT(*) | ---------| 48382 | Query execution time is more than 7 seconds Data sample: ID |APPLICATION_ID_NAME |AVG_FAILURE_DURATION |AVG_SUCCESS_DURATION |DEST= _NE_NAME |DESTINATION_HOST |DESTINATION_REALM_NAME |NUM_OF_RETRANSMISSION= _FRAMES |NUMBER_OF_REQUESTS |NUMBER_OF_RESPONSES |ORIGIN_HOST |ORIGIN_REAL= M_NAME |PROCEDURE_DURATION |PROCEDURE_DURATION_COUNTER |PROCEDURE_DURATION= _MAX |PROCEDURE_SUBTYPE |PROCEDURE_TYPE |RELEASE_CAUSE |RELEASE_TYPE |SO= URCE_NE_NAME |START_TIME |TIME_STAMP |TRANSPORT_LAYER_PRO= TOCOL_NAME |VLAN_ID | ----|---------------------|---------------------|---------------------|----= ---------|------------------|------------------------|---------------------= --------|-------------------|--------------------|-------------|-----------= --------|-------------------|---------------------------|------------------= -----|-------------------|---------------|--------------|---------------|--= -------------|--------------------|--------------------|-------------------= ------------|--------| 0 |APPLICATION_ID_NAME9 |1 |6 |dest= Dest39 |DESTINATION_HOST5 |DESTINATION_REALM_NAME4 |1 = |1 |9 |ORIGIN_HOST3 |ORIGIN_REAL= M_NAME6 |6 |4 |8 = |PROCEDURE_SUBTYPE2 |17 |43 |RELEASE_TYPE7 |So= urceDest71 |2017-12-04 14:37:56 |2017-12-04 14:37:56 |TRANSPORT_LAYER_PRO= TOCOL_NAME1 |48 | 1 |APPLICATION_ID_NAME7 |7 |1 |dest= Dest56 |DESTINATION_HOST5 |DESTINATION_REALM_NAME8 |8 = |6 |4 |ORIGIN_HOST1 |ORIGIN_REAL= M_NAME4 |2 |29 |3 = |PROCEDURE_SUBTYPE2 |15 |20 |RELEASE_TYPE77 |So= urceDest33 |2017-12-04 14:37:57 |2017-12-04 14:37:57 |TRANSPORT_LAYER_PRO= TOCOL_NAME7 |47 | There are indexes: ID START_TIME START_TIME + SOURCE_NE_NAME + DEST_NE_NAME (CREATE INDEX DINAMICHPA_TIME_D= EST_SOURCE ON HPA (START_TIME,SOURCE_NE_NAME,DEST_NE_NAME)) [04-12-2017 13:09:44][INFO ][grid-nio-worker-tcp-comm-2-#27][TcpCommunicati= onSpi] Accepted incoming communication connection [locAddr=3D/10.0.5.236:47= 101, rmtAddr=3D/172.16.10.232:53754] [04-Dec-2017 13:09:52][WARN ][query-#290][IgniteH2Indexing] Query execution= is too long [time=3D7152 ms, sql=3D'SELECT COUNT(*) __C0_0 FROM PUBLIC.HPA __Z0 WHERE __Z0.DEST_NE_NAME =3D ?1', plan=3D SELECT COUNT(*) AS __C0_0 FROM PUBLIC.HPA __Z0 /* PUBLIC.DINAMICHPA_TIME_DEST_SOURCE: DEST_NE_NAME =3D ?1 */ WHERE __Z0.DEST_NE_NAME =3D ?1 , parameters=3D[destDest17]] [04-12-2017 13:10:27][INFO ][grid-timeout-worker-#23][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=3Dc4f42f84, uptime=3D00:37:00.639] ^-- H/N/C [hosts=3D2, nodes=3D2, CPUs=3D12] ^-- CPU [cur=3D0.13%, avg=3D4.76%, GC=3D0%] ^-- PageMemory [pages=3D818037] ^-- Heap [used=3D1655MB, free=3D59.53%, comm=3D4090MB] ^-- Non heap [used=3D63MB, free=3D95.82%, comm=3D65MB] ^-- Public thread pool [active=3D0, idle=3D1, qSize=3D0] ^-- System thread pool [active=3D0, idle=3D8, qSize=3D0] ^-- Outbound messages queue [size=3D0] [04-12-2017 13:10:27][INFO ][grid-timeout-worker-#23][IgniteKernal] FreeLis= t [name=3Dnull, buckets=3D256, dataPages=3D600522, reusePages=3D0] ---------------------------------------------------------------------------= ------------------------------------------------------------------Client---= ------------------------------------------------------------------ Out put: [15:09:44] Topology snapshot [ver=3D6, servers=3D1, clients=3D1, CPUs=3D12,= heap=3D5.8GB] SQL Result: [[48382]] GET in sec: 7377 ---------------------------------------------------------------------------= ----------code----------------------------------------------------------- /* * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance with * the License. You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ package org.apache.ignite.examples.persistentstore; import org.apache.ignite.Ignite; import org.apache.ignite.IgniteCache; import org.apache.ignite.IgniteDataStreamer; import org.apache.ignite.Ignition; import org.apache.ignite.cache.CacheAtomicityMode; import org.apache.ignite.cache.CacheWriteSynchronizationMode; import org.apache.ignite.cache.query.QueryCursor; import org.apache.ignite.cache.query.SqlFieldsQuery; import org.apache.ignite.configuration.CacheConfiguration; import org.apache.ignite.examples.datagrid.CacheQueryExample; import org.apache.ignite.examples.model.Organization; import org.apache.ignite.configuration.CollectionConfiguration; import java.util.List; /** * This example demonstrates the usage of Apache Ignite Persistent Store. *

* To execute this example you should start an instance of {@link Persistent= StoreExampleNodeStartup} * class which will start up an Apache Ignite remote server node with a prop= er configuration. *

* When {@code UPDATE} parameter of this example is set to {@code true}, the= example will populate * the cache with some data and will then run a sample SQL query to fetch so= me results. *

* When {@code UPDATE} parameter of this example is set to {@code false}, th= e example will run * the SQL query against the cache without the initial data pre-loading from= the store. *

* You can populate the cache first with {@code UPDATE} set to {@code true},= then restart the nodes and * run the example with {@code UPDATE} set to {@code false} to verify that A= pache Ignite can work with the * data that is in the persistence only. */ public class HPAPersistentStoreExample { /** Organizations cache name. */ private static final String ORG_CACHE =3D CacheQueryExample.class.getSi= mpleName() + "HPA"; /** */ private static final boolean UPDATE =3D true; /** * @param args Program arguments, ignored. * @throws Exception If failed. */ public static void main(String[] args) throws Exception { Ignition.setClientMode(true); try (Ignite ignite =3D Ignition.start("D:/apache-ignite-2.3.0-src/e= xamples/config/persistentstore/example-persistent-store.xml")) { // Activate the cluster. Required to do if the persistent store= is enabled because you might need // to wait while all the nodes, that store a subset of data on = disk, join the cluster. ignite.active(true); CacheConfiguration cacheCfg =3D new CacheConfigurati= on<>(ORG_CACHE); cacheCfg.setSqlSchema("PUBLIC"); cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); cacheCfg.setBackups(1); cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationM= ode.FULL_SYNC); cacheCfg.setStatisticsEnabled(true); cacheCfg.setIndexedTypes(Long.class, HPA.class); IgniteCache cache =3D ignite.getOrCreateCache(cacheC= fg); if (UPDATE) { System.out.println("Populating the cache..."); try (IgniteDataStreamer streamer =3D ignite.data= Streamer(ORG_CACHE)) { streamer.allowOverwrite(true); for (long i =3D 0; i < 5000000; i++) { streamer.addData(i, new HPA(i)); if (i > 0 && i % 10_000 =3D=3D 0) System.out.println("Done: " + i); } } } // Run SQL without explicitly calling to loadCache(). long s =3D System.currentTimeMillis(); QueryCursor> cur =3D cache.query( new SqlFieldsQuery("select count(*) FROM HPA where DEST_NE_= NAME =3D ?") .setArgs("destDest17")); System.out.println("SQL Result: " + cur.getAll()); long d =3D System.currentTimeMillis() - s; System.out.println("GET in sec: " + d); // Run get() without explicitly calling to loadCach e(). HPA hpa =3D cache.get(54321l); System.out.println("GET Result: " + hpa); System.out.println(cache.metrics().getOffHeapAllocatedSize()); System.out.println(cache.metrics().getSize()); System.out.println(cache.metrics().getOffHeapBackupEntriesCount= ()); System.out.println(cache.metrics().getOffHeapGets()); System.out.println(cache.metrics().getOffHeapHits()); System.out.println(cache.metrics().getOffHeapMisses()); System.out.println(cache.metrics().getOffHeapPuts()); System.out.println(cache.metrics().getOffHeapEvictions()); System.out.println(cache.metrics().getOffHeapHitPercentage()); } } } Regards, Sasha Haykin Senior BI Developer T. +972-77-7745-163 M. +972-54-7939-700 [cid:image002.png@01D36D18.F1C4B700] --_000_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

I Working on big POC for telecom industry solution


I have loaded to ignite 5,000,000 objects/rows (I loaded it in the client s= ide Please find the code below)

 

In the metric the used of heap looks too small (for = 5M rows) is it normal utilization?

 

[04-12-2017 13:10:27][IN= FO ][grid-timeout-worker-#23][IgniteKernal]

Metrics for local node (= to disable set 'metricsLogFrequency' to 0)

    ^-- N= ode [id=3Dc4f42f84, uptime=3D00:37:00.639]

    ^-- H= /N/C [hosts=3D2, nodes=3D2, CPUs=3D12]

    ^-- C= PU [cur=3D0.13%, avg=3D4.76%, GC=3D0%]

    ^-- P= ageMemory [pages=3D818037]

    ^-- H= eap [used=3D1655MB, free=3D59.53%, comm=3D4090MB]

    ^-- N= on heap [used=3D63MB, free=3D95.82%, comm=3D65MB]

    ^-- P= ublic thread pool [active=3D0, idle=3D1, qSize=3D0]

    ^-- S= ystem thread pool [active=3D0, idle=3D8, qSize=3D0]

    ^-- O= utbound messages queue [size=3D0]

[04-12-= 2017 13:10:27][INFO ][grid-timeout-worker-#23][IgniteKernal] FreeList [name= =3Dnull, buckets=3D256, dataPages=3D600522, reusePages=3D0]  &nbs= p; 

 

When I’m querying the data VIA JDBC the perfor= mance is very bad (I’m comparing it to VERTICA DB)

 

What I’m doing wrong?

 

The success factors for the POC is to  query fr= om  100M rows of data (with filters on dates+ filter on one elemen= t ) and get results in 3 seconds is it possible to achieve it with Ignite?<= o:p>

 

Here are the example o= f the data + query:
SELEC= T = count(*) FROM HPA WHERE DEST_NE_NAME=3D'destDest17' ;

COUNT(*) |

---------|

48382    |

Query execution time is more than 7 second= s

 

Data sample:

 <= /span>

 <= /span>

ID  |APPLICATION_ID_NAME  |AVG_FAILURE_DUR= ATION |AVG_SUCCESS_DURATION |DEST_NE_NAME |DESTINATION_HOST  |DESTINAT= ION_REALM_NAME  |NUM_OF_RETRANSMISSION_FRAMES |NUMBER_OF_REQUESTS |NUM= BER_OF_RESPONSES |ORIGIN_HOST  |ORIGIN_REALM_NAME  |PROCEDURE_DUR= ATION |PROCEDURE_DURATION_COUNTER |PROCEDURE_DURATION_MAX |PROCEDURE_SUBTYPE&nbs= p; |PROCEDURE_TYPE |RELEASE_CAUSE |RELEASE_TYPE   |SOURCE_NE_NAME= |START_TIME          |TIME_ST= AMP          |TRANSPORT_LAYER_= PROTOCOL_NAME  |VLAN_ID |

----|---------------------|---------------------|---= ------------------|-------------|------------------|-----------------------= -|-----------------------------|-------------------|--------------------|--= -----------|-------------------|-------------------|-----------------------= ----|-----------------------|-------------------|---------------|----------= ----|---------------|---------------|--------------------|-----------------= ---|-------------------------------|--------|

0   |APPLICATION_ID_NAME9 |1  &n= bsp;            = ;     |6        = ;            |destDe= st39   |DESTINATION_HOST5 |DESTINATION_REALM_NAME4 |1  =             &nb= sp;            = |1            =       |9       =             |ORIGIN_= HOST3 |ORIGIN_REALM_NAME6 |6        = ;          |4             = ;            &n= bsp;|8           &nb= sp;          |PROCEDURE_SUBTYP= E2 |17           &nb= sp; |43            |= RELEASE_TYPE7  |SourceDest71   |2017-12-04 14:37:56 |2017-12= -04 14:37:56 |TRANSPORT_LAYER_PROTOCOL_NAME1 |48    &nb= sp; |

 

1   |APPLICATION_ID_NAME7 |7  &n= bsp;            &nbs= p;    |1        = ;            |destDe= st56   |DESTINATION_HOST5 |DESTINATION_REALM_NAME8 |8  =             &nb= sp;            = |6            =       |4       =             |ORIGIN_= HOST1 |ORIGIN_REALM_NAME4 |2        = ;          |29            = ;             |= 3            &n= bsp;         |PROCEDURE_SUBTYPE2 |1= 5             |= 20            |RELEA= SE_TYPE77 |SourceDest33   |2017-12-04 14:37:57 |2017-12-04 14:37:= 57 |TRANSPORT_LAYER_PROTOCOL_NAME7 |47      |=

 

There are indexes:

ID

START_TIME

START_TIME + SOURC= E_NE_NAME + DEST_NE_NAME  (CREATE INDEX DINAMICHPA_TIME_DEST_SOURCE ON HPA (START_TIME,SOURCE_NE_NAME,DEST_NE_NAME))

 


[04-12-2017 13:09:44][INFO ][grid-nio-worker-tcp-comm-2-#27][TcpCommunicati= onSpi] Accepted incoming communication connection [locAddr=3D/10.0.5.236:47= 101, rmtAddr=3D/172.16.10.232:53754]

[04-Dec-2017 13:09:52][WARN ][query-#290][IgniteH2In= dexing] Query execution is too long [time=3D7152 ms, sql=3D'SELECT

COUNT(*) __C0_0

FROM PUBLIC.HPA __Z0

WHERE __Z0.DEST_NE_NAME =3D ?1', plan=3D<= /p>

SELECT

    COUNT(*) AS __C0_0

FROM PUBLIC.HPA __Z0

    /* PUBLIC.DINAMICHPA_TIME_DEST_SO= URCE: DEST_NE_NAME =3D ?1 */

WHERE __Z0.DEST_NE_NAME =3D ?1

, parameters=3D[destDest17]]

[04-12-2017 13:10:27][INFO ][grid-timeout-worker-#23= ][IgniteKernal]

Metrics for local node (to disable set 'metricsLogFr= equency' to 0)

    ^-- Node [id=3Dc4f42f84, uptime= =3D00:37:00.639]

    ^-- H/N/C [hosts=3D2, nodes=3D2, = CPUs=3D12]

    ^-- CPU [cur=3D0.13%, avg=3D4.76%= , GC=3D0%]

    ^-- PageMemory [pages=3D818037]

    ^-- Heap [used=3D1655MB, free=3D5= 9.53%, comm=3D4090MB]

    ^-- Non heap [used=3D63MB, free= =3D95.82%, comm=3D65MB]

    ^-- Public thread pool [active=3D= 0, idle=3D1, qSize=3D0]

    ^-- System thread pool [active=3D= 0, idle=3D8, qSize=3D0]

    ^-- Outbound messages queue [size= =3D0]

[04-12-2017 13:10:27][INFO ][grid-timeout-worker-#23= ][IgniteKernal] FreeList [name=3Dnull, buckets=3D256, dataPages=3D600522, r= eusePages=3D0]

----------------------------------------------------= ---------------------------------------------------------------------------= --------------Client-------------------------------------------------------= --------------

Out put:

[15:09:44] Topology snapshot [ver=3D6, servers=3D1, = clients=3D1, CPUs=3D12, heap=3D5.8GB]

SQL Result: [[48382]]

GET in sec: 7377

----------------------------------------------------= ---------------------------------code--------------------------------------= ---------------------

/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements.  See the NOTICE file distributed wit= h
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0 * (the "License"); you may not use this file except in compliance= with
* the License.  You may obtain a copy of the License at
*
*      http://www.apache.org/licenses/LICENSE-2.0<= br> *
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASI= S,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.<= br> * See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.ignite.examples.persistentstore;

import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.IgniteDataStreamer;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.CacheAtomicityMode;
import org.apache.ignite.cache.CacheWriteSynchronizationMode;
import org.apache.ignite.cache.query.QueryCursor;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.examples.datagrid.CacheQueryExample;
import org.apache.ignite.examples.model.Organization;
import org.apache.ignite.configuration.CollectionConfiguration;
import java.util.List;

/**
* This example demonstrates the usage of Apache Ignite Persistent Store. * <p>
* To execute this example you should start an instance of {@link Per= sistentStoreExampleNodeStartup}
* class which will start up an Apache Ignite remote server node with a prop= er configuration.
* <p>
* When {@code UPDATE} parameter of this example is set to {@code = true}, the example will populate
* the cache with some data and will then run a sample SQL query to fetch so= me results.
* <p>
* When {@code UPDATE} parameter of this example is set to {@code = false}, the example will run
* the SQL query against the cache without the initial data pre-loading from= the store.
* <p>
* You can populate the cache first with {@code UPDATE} set to {@c= ode true}, then restart the nodes and
* run the example with {@code UPDATE} set to {@code false} to= verify that Apache Ignite can work with the
* data that is in the persistence only.
*/
public class HPAPersistentStoreExample {
    /** Organizations cache name. */
    private static final String ORG_CACHE =3D CacheQueryExample.class"HPA";

    /** */
    private static final boolean UPDATE =3D true;

    /**
     * @param args Program arguments, ignored.
     * @throws Exception If failed.
     */
    public static void main(String[] args) throws Exception {
        Ignition.setClientMode(true);

        try (Ignite ignite =3D Ignition.start(&qu= ot;D:/apache-ignite-2.3.0-src/examples/config/persistentstore/example-persi= stent-store.xml")) {
            <= i>// Activate the cluster. Required to do if the persistent store is en= abled because you might need
            // to wa= it while all the nodes, that store a subset of data on disk, join the clust= er.
            <= /i>ignite.active(true);

            CacheCon= figuration<Long, HPA> cacheCfg =3D new CacheConfiguration<>(ORG_CACHE);
            cacheCfg= .setSqlSchema("PUBLIC");
            cacheCfg= .setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
            cacheCfg= .setBackups(1);
            cacheCfg= .setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
            cacheCfg= .setStatisticsEnabled(true);
           
            cac= heCfg.setIndexedTypes(Long.class, HPA.= class);

            IgniteCa= che<Long, HPA> cache =3D ignite.getOrCreateCache(cacheCfg);


            <= b>if (UPDATE) {
            &nb= sp;   System.out.println(= "Populating the cache...");

            &nb= sp;   try (IgniteDataStreamer<Long, HPA> streamer =3D ignite.da= taStreamer(ORG_CACHE)) {
            &nb= sp;       streamer.allowOverwrite(<= span style=3D"font-size:9.0pt;font-family:"Courier New";color:nav= y">true);

            &nb= sp;       for (long i =3D 0; i < 5000000; i++) {
            &nb= sp;           streamer.ad= dData(i, new HPA(i));

            &nb= sp;           <= span style=3D"font-size:9.0pt;font-family:"Courier New";color:nav= y">if (i > 0 && i % 10_000 =3D=3D 0)
            &nb= sp;            =    System.out.println("Done: " + i);
            &nb= sp;       }
            &nb= sp;   }
            }

            <= i>// Run SQL without explicitly calling to loadCache().
            <= /i>long s =3D System.currentTimeMillis();
            QueryCur= sor<List<?>> cur =3D cache.query(
            &nb= sp;   new SqlFieldsQuery("select count(*) FROM HP= A where DEST_NE_NAME =3D ?")
            &nb= sp;       .setArgs("destD= est17"));

            System.<= /span>out.println("SQL Result: " + cur.getAll());

           long d =3D System.currentTimeMillis() - s;
            System.<= /span>out.println("GET in sec: " + d);
            <= i>// Run get() without explicitly calling to loadCach e().
            <= /i>HPA hpa =3D cache.get(54321l);

            System.<= /span>out.println("GET Result: " + hpa);

            System.<= /span>out.println(cache.metrics().getOffHe= apAllocatedSize());
            System.<= /span>out.println(cache.metrics().getSize(= ));
            System.<= /span>out.println(cache.metrics().getOffHe= apBackupEntriesCount());
            System.<= /span>out.println(cache.metrics().getOffHe= apGets());
            System.<= /span>out.println(cache.metrics().getOffHe= apHits());
            System.<= /span>out.println(cache.metrics().getOffHe= apMisses());
            System.<= /span>out.println(cache.metrics().getOffHe= apPuts());
            System.<= /span>out.println(cache.metrics().getOffHe= apEvictions());
            System.<= /span>out.println(cache.metrics().getOffHe= apHitPercentage());

        }
    }
}

 

 

 

Regards,

 

Sasha= Haykin

Senior BI Developer

T. +972-77-7745-163

M. +972-54-7939-700

 

--_000_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_-- --_004_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_ Content-Type: image/png; name="image002.png" Content-Description: image002.png Content-Disposition: inline; filename="image002.png"; size=3858; creation-date="Mon, 04 Dec 2017 14:00:04 GMT"; modification-date="Mon, 04 Dec 2017 14:00:04 GMT" Content-ID: Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAKkAAAAeCAYAAABXGj7iAAAAAXNSR0IArs4c6QAAAAlwSFlzAAAO xAAADsQBlSsOGwAAABl0RVh0U29mdHdhcmUATWljcm9zb2Z0IE9mZmljZX/tNXEAAA6SSURBVHhe 7VsLcFTVGT7n3HPvPiDJxiUaDCE+GDutaam1WgOmjgbUyqijtSh1VLQtttSZqq0OVmHdAcfiE7VV rFqtFUdbR63FJ9KRRoiPUWuLotRHIYlGIWQTQ/Z17zn9/rNZTArs3mWSGuuemeiSPf95/P/3v09k PB5n5VHmwFjmgBzLhyufrcwB4kAZpGUcjHkOlEE65kVUPmAZpGUMjHkOjCmQxiIt9WJ88FDmZsA4 YZinmHw+3vX41jHPyfIBR40DYwaksUhE8qD1exYefzRz3dyFpWSir+cBfJozahwoLzzmOTBmQCrC h53LZOBo9kmCMT3IN85hUK0z4rXHPRjrevrRMc/N8gFHhQNjAqSxyGl1QvXHmacAUCDUdnJAddOw pgH4fG8pQoE18cTqnlHhQnnRMc2BMQFSEehbwgKhiSydNADVmXQrEzzMZeBQlkV8GggdJPTAL8HJ S8Y0N8uHGxUOfOYgXVx73LGMi7NZJgXXjmTJy6YsLi6AJa1Uyn2OcW6xLCyqZV2wuLbloYVdq18c FU6UFx2zHPhMQRqrnTaea30Nty3Bsh5jTpCx1MDyhR+t+gdxLL7PjD+w4Li5xsI6waCXUdfFIme0 xBMPUPpfHl8QDsjWTX0HhJ3wAa7bz1yEhKUMIQVFjkmXyfeb68IflkJLcwUbdxELBKcaKyptBjD+ W3Hnqvw6Kq0XCZ48gVlyb5rDneCRnHX/GN/f7HevFd3d9pQBdpxiVlAwc17fA+xIgiddm8dn/3lm NJr1TYiJ69d/2eqtevEYwdShTPMaxhku6GNoloX3+NBV2ZeaG6J/80GxY0pbW0SyfTdPQx/xYOay iBKK6ni+74zpGhR9QrG3XPnJ2ua6OgjG32jb1H0Yktxf4OwceYUFqtcqejuubmxshPUpbbS1J07T nJ9JfhVr9UlLeOdBfJertMNsSlpKHFor0LntbZsSNzc1RK7zSw7XfTDjuJQL2VMWDykiQVoU//iZ HTVRJErtyOwXA6S3MA9lKfrhYmFswlEr41vXvOdnr8mp0DjF0vdLO1BBZy1lWDgSz7rqwJT9Ylt7 95Km+ugTfujbUE5jkRfusC1rLudwVnS9UgbEICDaFzYl7s7K7fP9gKW1s3O83Ddxr5DWKcICRggm eziUgkp74VWt3WpOc1R0+1pGWA3SCcwmWQKlwJY+tb+qrnF9Z+f5jXV1vb7WwCQAdAFwvsR2HEsI C3ZrQEnwz/UgfPrROBwpgmWT0hfjrAZmQANgW5asVxa7tq2z7+Wmuso1xQ4UW7SI89vblnLHqWQZ cuUhJEvJZ3T0gxXs4xx1LBbjeKGlVar6Dq63nsmd0BEmNg0EJwilr8aU04vtM/g9qV4SZwVIvdz9 yGoXuh4I6F6K6rWcC8xv8rL64bb2vlOb6iuLA/X1zUdwi88lYWvlwsBY+PEXWSnIQXk4J+JzEbDP ld442u+hYneVmYoTZUieQjJxkWzSfgJRVCmD5E/0xCPbCc3k/X1zWDTyaz9rYKesm0nrQUtqDA9A e3p/RkyCt57b3FD5TqF1HmsNhGv2+/gmS8ofEt+y6bThGxbqJTUn2OdKPxgQTtLNZCg5KWamUSdi 38RhQh6sIQk+m8l+C78rClJxZ9v3mR2YxTIAHbQFhANaqgXxDRuMqYNVXnD8eReRVJcg/kzD6i6A 8FZBcrahkXJ2vHYmaqerHvbDQLpW3uvhfpBDZj1+U8h9WyCIQvkmE1/AfLpfwMtmbmjtHGhFaPNJ oX3hZqdK3CsPNuy43vXUGljygqiBXMlKHAmATiXAQPvhXXijH5AyqQ7JiRF05HE9d4Or9HsINYpZ m9xVkBxg7mR8+iopqFbwtJwf4pO/O08zfMsginOmIxl+GgbsHBiw53e13sudAw01+/XeJR27xctm jYEYOoapN2kvbtf+bkgd6ycGW7e593oprYtJ+wZHUXMRq51VI1T2qlx4SDVRJEvp5G3xjtWvGYB2 dn8NiFgIpyGggU9AA19FRr8mvs/Mu1kwPM8kUdzIemks0PJcPL16m19G5sIl3e9y55jmSeGCdK3v dO/NA+xESPg68CVCisgt60tSZWYwFn6k0J44XQUMsNFzcllK6Web6qsu8nPOto7e620nONVDaxhW hXkZl4xB8cF1JTWRaZg9Pfe2pknVtxQn/HQGjMM5XIp7SEEIKJzr8aXQD51LRksB6GTV8fkAeOqV a9u7z59eH31w6Ly2TX1NTKh7pOMcRKDOezridyFQ8cmpFPn7ookC1+5GrT81DgTxYpcSLBNjgXCD AVs+WUp5S4kOVqpKKnGnEwyFaSmdSt75cnd3y2HRaA8SqkUiPXA8fMhk09sPhKeI6uQVrItdXGzP nb5PFn+i2DwlSoHHXUgIJEC6PC98T+tp+FwQpBAOYDIkT4Ln831GrXsgoM2uERJUROstvmg1Bb9D hihhzzzZntD81+HIbAPeGdzhaXycKaQMEuDguqu4kivWbko0TG+IXENkiD/ncEv8Rki7mgBK7h3K pQHov+DzSYVAauThhzFKoKLpZ+LgHLjtZpxknokt88kS0zEkSEYQMpM5QTjWXpl08q0cKESVOyBm sSi7D49MPoKLvwy/XmFozRrW/MHa6boSjsGCIX/3ozVdGfoTrOcSaPgEY12Y3r/YXoJRtPDpMGQ+ h9vr3Cir2XIvrLSd4tyV2wcYi/igRtg2dCBw8EE0nAQ0JYaxu9kCGYpS1yH9upV73u/g8ieSRYWy W7ZjLV3XkajHaT+Ct4lzwQW5eMqDANA+JfhPcIzXQ1KelPfQO2k45QvNk+r6fV1QsUpOqdfgQNi9 W8agvhmw9LZrYD1zcSVqokiWntXVHffBGppRsd15dGJl8qGNGzcahk856CC+9sOQZPW57xGD3n/l PjNm82D4ZJZGdcQJBHTGuzYWOfyYeOIlLDryo6eHJWuq2FYoxgRj3ZWOUvbelEjsiHFGctfmxvB2 rIcfOBMz8v8fyV1Gey0TXYem7x95qrWzbyZ8/p1Ioo5ATG/idGnZF9AJ6LP5t+MgbM1uyHr8R80N VWsR5n1jqCMaBtLBgHVCW3vvlQjYCwgdsY9gjrD0WYr67flBSdBuhgh2z9eUoZvOEua5me3Qh0vy yRKRNTaGEQOEWVNT045VTooODzs0z17KM+mjEOhECOzaCU4TrGo+CG4cDdZXV1eD4R/hAQENqknx gacmTlQA6Whs93+zJnlZukxzXeUbCONm6Wx6mWXZZ1Gcmo83KW6HlYVTzDxpW968IxqiHUQDUA5z 0MMtqQmWRRSBbqwot0xZM6cJn9ZzxLu7ols8oWUKfPflps5Jgx6QpAduiHet/nvRff5rQrxrzUa4 /RizAjdh83zt9Aq0V1cu7Hr6X6WuV2y+dLfsrYWuoeQXrommvzFUsYrRl78noFKSGj4bMeibCJuu QpwqchUlslWZG/k47zLkHbs1irsM6E2Wb4qyWAeZHsUTQ8sC9D1pgIW9cjVHzrKp1Gol+h9nDEnm kBELzApxmb2NV9VE88mS7t32pubOtXsqQJXqvRUxzcm8asIx8BMUOuzlJbYsR+XgRMSuiOFGdEwX 3BpHHKUaJpT8pRFd/Qu0WFN95FcA6tuIV5fT2wyUy37WNDny22IsGF6CypVotmUzyduxCFJorvDv BiQw55FpphocARTF1g3ZdOpeLB4AlC1P6W5nfOW9TdGqnWPZ6lSEa+sVltj6CggBKBv1N+/P8a5n CtYaCx0c8acbm9Qyn/cn5uJQnA30U5mP61SqGnQjC1KhThdCclOY12qbFPYua33FGO33+8fWD4Qm BpPBVCikgsmkSIVSST8dJ7/rf9bzANRHEHNSt7CyuSHS6uc8wy0pZc1abUF9jZ7F7RhA//soCSw2 ZTiYaSRp+2qV6UCb8L5im8ClU09/QbF5pX6PuurboKFsf9TG2vbEDNQvvkPdOFP389Qzh9WFB9O8 0dm2psq9UFmBH9huWqlwwJKevQw7lVTvHJ2TjdyqqH2/XspqO7l74FS0DgyMaw6bLNMMoH/Jus09 QSntyynwBVKr0JO+F+CtxnefKwaSGqZ6evpZXeGsGR2SryO9ux1xqEQSSW1j9H/l7aUwd8dczXxX HlBA3wuh1IFUJ6GyTDqZrN2TPelxzB7QDfdCJZTO9mAv3yS7LjL39HAWHi7EaZOrrwAoHViUSyg2 QwAMT89vRpeC4WHJ5wKog41fR1ZVnIw+/K5f+QtWAUQeDnCehQC/hmp1VCJBy/eP0yaOf84PZ9H8 zFiDfQ3TphS6EfvhrUHhtigm0p8mfMVFDTifkAKrvsIiCnbyCSztiRzv29iTDI3fUjZKlProfHs8 V8bmo1Jm88PDoXP8d0JyFvXSde29tpTyQnKBBqdS3IwugkAX4aZSN/+fzzc9Ybz453q3YQrF3lYw aB5a5AGKDPRtL+z83O95BfM6cq/VchUQ8OlYPDg51s/TJA2rnQPoYB+eCVOWKTZwta58+d48FBLi DPPjs6RvFJj2Np7SvKvBf9xRDW2K3Sn/PUrx2sq90gFTqQabVU6hXsU09KDbNveELMc5f5AZ9OJm GbTWxgsh30/1/B5wJOZBTqjpwrYhx8oJoLDkdrwECgRhQVOvIwjHk7Vwp9+zuCLUKr1slx0M1VI3 xTQK/aIFiSnVVSxbUsVkixt2ij7YMfgS4knsFbMDQTvXqck/GPJ7asMY83rKAq9MW1azlX6pAWlo t2UcLJJMqg/5teA7byGhY0Ze9BJNOdJTvB/N0m1Gew0j0Suuri7Yg6/o6/xpf2QSuCm+q0gI+BMP HO4yALULQC2aTPm9+AjNI/v5Me6HO/t8T2rKb7xHuekHpHCWIVkq6e/+URf8ADHt99x05kKYxUOg G5GS7qJZH14hvQblWAblaPdDi6L5q/Bos1EunAfjQS+nUDbbo5EEyjaAV/fAc/7V7wp4kJhCmNRN KDWMVsJ3HL6LPbJIUnOYZCzxH81MPjp9LKJqAAAAAElFTkSuQmCC --_004_HE1PR05MB09721EA6598F04C60A48D881A33C0HE1PR05MB0972eurp_--