ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexei Scherbakov <alexey.scherbak...@gmail.com>
Subject Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?
Date Wed, 25 May 2016 10:23:26 GMT
For postgres test I mean initial jdbc query and result set traversal.
For Ignite I mean sql query and iterator traversal.
Also it would be interesting to see result of
*SELECT count(*) from the query above in both cases.*

2016-05-25 12:00 GMT+03:00 Tomek W <rrrtomtomrrr@gmail.com>:

> [image: Obraz w treści 1]
>
> What code do you mean ? JDBC client ?
>
> 2016-05-25 10:25 GMT+02:00 Alexei Scherbakov <alexey.scherbakoff@gmail.com
> >:
>
>> What's the batch size for postgresql ?
>> What's the size of one entry ?
>> Could you provide the test code for both postgres and Ignite (just the
>> query + read with the time estimation) ?
>>
>> 2016-05-25 11:13 GMT+03:00 Tomek W <rrrtomtomrrr@gmail.com>:
>>
>>> | How many entries are downloaded to the client in both cases?
>>> 3000 000
>>>
>>> | Do the both queries involve network I/O ?
>>> No, I have only local one server (for testing purpose).
>>>
>>>
>>> 2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <
>>> alexey.scherbakoff@gmail.com>:
>>>
>>>> SELECT * is not really a good test query.
>>>> It's result can be affected not only by engine performance.
>>>>
>>>> How many entries are downloaded to the client in both cases?
>>>> Do the both queries involve network I/O ?
>>>>
>>>> 2016-05-25 7:58 GMT+03:00 Denis Magda <dmagda@gridgain.com>:
>>>>
>>>>> In general Ignite is designed to be used in a distributed environment
>>>>> when gigabytes or terabytes of dataset is spread across many cluster
nodes
>>>>> and SQL queries executed across the cluster should be faster since
>>>>> resources of all the machines will be used and as a result a query should
>>>>> be completed quicker. In your scenario you just have only a single cluster
>>>>> node and in fact comparing performance of PostgreSQL and H2 (engine that
is
>>>>> used by Ignite SQL) and I can consider that Ignite SQL can work slightly
>>>>> slowly but this in is not Ignite usage scenario.
>>>>>
>>>>> However if you try to create a cluster of several nodes running on
>>>>> different physical machines, pre-load gigabytes of data there and compare
>>>>> Ignite SQL and PostgresSQL you should see performance improvements on
>>>>> Ignite side.
>>>>>
>>>>> In any case taking into account the advise above do the following:
>>>>> - execute “EXPLAIN” query to see that the index is chose properly
[1];
>>>>> - H2 console will allow you to see how fast a query is presently
>>>>> executed on a single node removing several Ignite layers [2];
>>>>> - check if you have any GC pauses during query execution since it can
>>>>> affect execution time [3]
>>>>>
>>>>> Also share the objects you use as keys and values.
>>>>>
>>>>> [1] https://apacheignite.readme.io/docs/sql-queries#using-explain
>>>>> [2]
>>>>> https://apacheignite.readme.io/docs/sql-queries#using-h2-debug-console
>>>>> [3]
>>>>> https://apacheignite.readme.io/v1.6/docs/jvm-and-system-tuning#section-detailed-garbage-collection-stats
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On May 25, 2016, at 3:23 AM, Tomek W <rrrtomtomrrr@gmail.com> wrote:
>>>>>
>>>>>
>>>>> +==============================================================================================+
>>>>> |     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time
>>>>> |  Size   | Hi/Mi/Rd/Wr |
>>>>>
>>>>> +==============================================================================================+
>>>>> | 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49
>>>>> | 3000000 | Hi: 0       |
>>>>> |                          |      |           |          |
>>>>> |         | Mi: 0       |
>>>>> |                          |      |           |          |
>>>>> |         | Rd: 0       |
>>>>> |                          |      |           |          |
>>>>> |         | Wr: 0       |
>>>>>
>>>>> +----------------------------------------------------------------------------------------------+
>>>>>
>>>>>
>>>>> I followed your hints. Actually, client doesn't require such many
>>>>> memory as before - thanks for it.
>>>>>
>>>>>
>>>>> When it comes to configuration of server, I also followed your hints,
>>>>> results:
>>>>>
>>>>> Querying is done by JDBC Client.  In ignite and postgresql I have
>>>>> single index on column A.
>>>>>
>>>>> Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
>>>>> Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.
>>>>>
>>>>> As you  can see, postgres is still bettter than Ignite.  I show you
>>>>> significant fragments of my configuration:
>>>>> http://pastebin.com/EQC4JPWR
>>>>>
>>>>> And xml for server file:
>>>>> http://pastebin.com/enR9h5J4
>>>>>
>>>>>
>>>>> Try to consider why postgresql is still better, please.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Best regards,
>>>> Alexei Scherbakov
>>>>
>>>
>>>
>>
>>
>> --
>>
>> Best regards,
>> Alexei Scherbakov
>>
>
>


-- 

Best regards,
Alexei Scherbakov

Mime
View raw message