ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JohnnyS <jtk...@gmail.com>
Subject Local Node Query Optimization
Date Mon, 17 Aug 2015 11:45:51 GMT

I'm using Apache Ignite to query a large spatial database using a z-order
curve and I'm trying to improve the query performance in local node, i.e. I
use sql.setLocal(true). The database consists of GPS tracks from multiple
users divided into tiles which I query. The data is meant to be mostly
read-only with updates once a day at most. I use partitioned mode.

I query the cache for tracks in a tile and the program needs to iterate over
the result set as quickly as possible or calculate the total number of users
in a tile. I'm using a group index (index of the tile, id for user) which is
used in the query (I can see this in H2 console). The records in a tile are
sorted by the user id.

For a test tile that I've been using, the index limits the query to 33000
records from which the total number of users needs to be calculated.
Currently the query takes 120 ms when using ONHEAP_TIERED cache and a single
node. For OFFHEAP_TIERED cache the performance drops to 200 ms. The
performance scales linearly with the number of nodes when using a broadcast
which is good. I however intend to use the cache as a part of a web
application and the single node performance needs to be faster. In a C++
implementation that I did previously (and which is hard to scale to multiple
nodes and large data sets), a similar calculation can be done in a few ms
when using cache friendly containers.

My question is is there anything I can do to increase the performance of the
local node queries? I've been trying most of the performance tips from
http://apacheignite.gridgain.org/v1.3/docs/performance-tips except for the
externalizable part. For example, can I change H2 settings somehow or use
something similar to local peek to increase the query performance?


View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Local-Node-Query-Optimization-tp996.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message