ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomek W <rrrtomtom...@gmail.com>
Subject Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?
Date Wed, 25 May 2016 09:00:30 GMT
[image: Obraz w treści 1]

What code do you mean ? JDBC client ?

2016-05-25 10:25 GMT+02:00 Alexei Scherbakov <alexey.scherbakoff@gmail.com>:

> What's the batch size for postgresql ?
> What's the size of one entry ?
> Could you provide the test code for both postgres and Ignite (just the
> query + read with the time estimation) ?
>
> 2016-05-25 11:13 GMT+03:00 Tomek W <rrrtomtomrrr@gmail.com>:
>
>> | How many entries are downloaded to the client in both cases?
>> 3000 000
>>
>> | Do the both queries involve network I/O ?
>> No, I have only local one server (for testing purpose).
>>
>>
>> 2016-05-25 9:59 GMT+02:00 Alexei Scherbakov <alexey.scherbakoff@gmail.com
>> >:
>>
>>> SELECT * is not really a good test query.
>>> It's result can be affected not only by engine performance.
>>>
>>> How many entries are downloaded to the client in both cases?
>>> Do the both queries involve network I/O ?
>>>
>>> 2016-05-25 7:58 GMT+03:00 Denis Magda <dmagda@gridgain.com>:
>>>
>>>> In general Ignite is designed to be used in a distributed environment
>>>> when gigabytes or terabytes of dataset is spread across many cluster nodes
>>>> and SQL queries executed across the cluster should be faster since
>>>> resources of all the machines will be used and as a result a query should
>>>> be completed quicker. In your scenario you just have only a single cluster
>>>> node and in fact comparing performance of PostgreSQL and H2 (engine that
is
>>>> used by Ignite SQL) and I can consider that Ignite SQL can work slightly
>>>> slowly but this in is not Ignite usage scenario.
>>>>
>>>> However if you try to create a cluster of several nodes running on
>>>> different physical machines, pre-load gigabytes of data there and compare
>>>> Ignite SQL and PostgresSQL you should see performance improvements on
>>>> Ignite side.
>>>>
>>>> In any case taking into account the advise above do the following:
>>>> - execute “EXPLAIN” query to see that the index is chose properly [1];
>>>> - H2 console will allow you to see how fast a query is presently
>>>> executed on a single node removing several Ignite layers [2];
>>>> - check if you have any GC pauses during query execution since it can
>>>> affect execution time [3]
>>>>
>>>> Also share the objects you use as keys and values.
>>>>
>>>> [1] https://apacheignite.readme.io/docs/sql-queries#using-explain
>>>> [2]
>>>> https://apacheignite.readme.io/docs/sql-queries#using-h2-debug-console
>>>> [3]
>>>> https://apacheignite.readme.io/v1.6/docs/jvm-and-system-tuning#section-detailed-garbage-collection-stats
>>>>
>>>> —
>>>> Denis
>>>>
>>>> On May 25, 2016, at 3:23 AM, Tomek W <rrrtomtomrrr@gmail.com> wrote:
>>>>
>>>>
>>>> +==============================================================================================+
>>>> |     Node ID8(@), IP      | CPUs | Heap Used | CPU Load |   Up Time
>>>> |  Size   | Hi/Mi/Rd/Wr |
>>>>
>>>> +==============================================================================================+
>>>> | 0F0AAF99(@n0), 127.0.0.1 | 8    | 54.50 %   | 3.23 %   | 00:13:13:49
>>>> | 3000000 | Hi: 0       |
>>>> |                          |      |           |          |
>>>> |         | Mi: 0       |
>>>> |                          |      |           |          |
>>>> |         | Rd: 0       |
>>>> |                          |      |           |          |
>>>> |         | Wr: 0       |
>>>>
>>>> +----------------------------------------------------------------------------------------------+
>>>>
>>>>
>>>> I followed your hints. Actually, client doesn't require such many
>>>> memory as before - thanks for it.
>>>>
>>>>
>>>> When it comes to configuration of server, I also followed your hints,
>>>> results:
>>>>
>>>> Querying is done by JDBC Client.  In ignite and postgresql I have
>>>> single index on column A.
>>>>
>>>> Ignite: SELECT * FROM table WHERE A > 1345 takes 6s.
>>>> Postgres: SELECT * FROM table WHERE A > 1345 takes 4s.
>>>>
>>>> As you  can see, postgres is still bettter than Ignite.  I show you
>>>> significant fragments of my configuration:
>>>> http://pastebin.com/EQC4JPWR
>>>>
>>>> And xml for server file:
>>>> http://pastebin.com/enR9h5J4
>>>>
>>>>
>>>> Try to consider why postgresql is still better, please.
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Best regards,
>>> Alexei Scherbakov
>>>
>>
>>
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>

Mime
View raw message