ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexei Scherbakov <alexey.scherbak...@gmail.com>
Subject Re: off heap indexes - setSqlOnheapRowCacheSize - how does it improve efficiency ?
Date Tue, 24 May 2016 13:22:42 GMT
Have you configured near cache on the client?
Do you buffer data somewere ?
Share the details of the loading process.


2016-05-24 16:07 GMT+03:00 Tomek W <rrrtomtomrrr@gmail.com>:

> Thanks for you answer.
>
>
> I didn't measure it. Simply, I was getting GC overflow error, so I
> succesively increased it.
> How to measure it ? Why client requires so much memory ?
>
> 2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <alexey.scherbakoff@gmail.com
> >:
>
>> Just use this:
>>
>> -server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
>> -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m
>> -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024
>> -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60
>> -XX:+DisableExplicitGC
>>
>> How do you measure client memory usage ?
>>
>> 2016-05-24 15:04 GMT+03:00 Tomek W <rrrtomtomrrr@gmail.com>:
>>
>>> Sorry,
>>> I made a mistake - I wanted to say that I am going to use ON_HEAP.
>>> Can you suggest my more details about tuning ?
>>> I have client (which run hot loading data from postgresql) and server
>>> node (keep cache - data).
>>> Now client also requires ~4GB data. Why ?   After all, it doesn't keep
>>> data, only run hot loading.
>>>
>>>
>>>
>>>
>>>
>>> 2016-05-24 13:44 GMT+02:00 Alexei Scherbakov <
>>> alexey.scherbakoff@gmail.com>:
>>>
>>>> Try to start with some larger number, if default value is too low for
>>>> you.
>>>> On example, set it to 50000, and see if the performance is OK.
>>>> If not, increase to 100000 etc.
>>>> I can't help you further without knowing your data access patterns.
>>>>
>>>> BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val
>>>> suggested.
>>>> Don't forget to tune GC as described here:
>>>>
>>>>
>>>> https://apacheignite.readme.io/docs/jvm-and-system-tuning#jvm-tuning-for-clusters-with-on_heap-caches
>>>>
>>>>
>>>> 2016-05-23 22:05 GMT+03:00 Tomek W <rrrtomtomrrr@gmail.com>:
>>>>
>>>>> Ok, I am going to use OFF_HEAP.
>>>>>
>>>>>
>>>>> On each node I am going to use about 10 GB.  (My ram is 16GB).
>>>>> Can you help me adjust configuration for this aim ?
>>>>> It is very important for me.
>>>>> Aim: Extremely fast sql quries.
>>>>>
>>>>>
>>>>> 2016-05-23 18:13 GMT+02:00 Alexei Scherbakov <
>>>>> alexey.scherbakoff@gmail.com>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Generally speaking, settings setSqlOnheapRowCacheSize to larger
>>>>>> value increases
>>>>>> SQL performance in OFFHEAP_TIERED mode, but also means more job for
>>>>>> GC,
>>>>>> so it should be used with care.
>>>>>>
>>>>>> The value should be set to the size of your application's
>>>>>> working(frequently accessed) data set.
>>>>>>
>>>>>> 2016-05-23 13:07 GMT+03:00 vkulichenko <valentin.kulichenko@gmail.com
>>>>>> >:
>>>>>>
>>>>>>> Are you using offheap? What is your data size?
>>>>>>>
>>>>>>> Generally, I would recommend to use on-heap with SQL queries
if this
>>>>>>> possible (unless you have a very big data sets and want to avoid
>>>>>>> having
>>>>>>> large heap sizes). If you still have to use offheap, you can
try
>>>>>>> playing
>>>>>>> with this parameter and see what performance you get with different
>>>>>>> values.
>>>>>>> The optimal value depends on a particular application.
>>>>>>>
>>>>>>> -Val
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html
>>>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Best regards,
>>>>>> Alexei Scherbakov
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Best regards,
>>>> Alexei Scherbakov
>>>>
>>>
>>>
>>
>>
>> --
>>
>> Best regards,
>> Alexei Scherbakov
>>
>
>


-- 

Best regards,
Alexei Scherbakov

Mime
View raw message