ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Off-Heap SQL Indexes
Date Tue, 22 Nov 2016 01:00:53 GMT
Prachi, thanks, looks good to be.

BTW, I’ve started refining the overall SQL documentation. At the moment I’ve created a
new section named “SQL” located right below “Data Grid” and that sections contains
a dedicated page about the indexes. So the new link to the page you’re referring to is the
following:
https://apacheignite.readme.io/docs/indexes#off-heap-sql-indexes <https://apacheignite.readme.io/docs/indexes#off-heap-sql-indexes>

—
Denis
 
> On Nov 21, 2016, at 12:32 PM, Prachi Garg <pgarg@gridgain.com> wrote:
> 
> Fixed. Please review - http://apacheignite.gridgain.org/v1.7/docs/sql-queries#off-heap-sql-indexes
<http://apacheignite.gridgain.org/v1.7/docs/sql-queries#off-heap-sql-indexes>
> 
> -P
> 
> On Mon, Nov 7, 2016 at 9:36 AM, Denis Magda <dmagda@gridgain.com <mailto:dmagda@gridgain.com>>
wrote:
> Sergi,
> 
>> May be you use
>> some weird email client which renders >> signs wrong?
> 
> Right, my Apple’s Mail agent omitted ‘>’ character from your original statement
applying some HTML formatting. 
> 
> Prachi, please update all the docs keeping in my that this sentence is technically correct
> 
>>> SQL engine will store indexes in the off-heap region if
>>> setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported
>>> off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0
>>> automatically if it’s value is less than -1 and one of off-heap memory
>>> modes is used.
> 
> —
> Denis
> 
>> On Nov 5, 2016, at 6:23 AM, Sergi Vladykin <sergi.vladykin@gmail.com <mailto:sergi.vladykin@gmail.com>>
wrote:
>> 
>> Your final statement seems correct to me, but I said exactly that, I don't
>> know where did you get that setOffHeapMaxMemory must be 0. May be you use
>> some weird email client which renders >> signs wrong?
>> 
>> Sergi
>> 
>> 2016-11-04 22:25 GMT+03:00 Denis Magda <dmagda@gridgain.com <mailto:dmagda@gridgain.com>>:
>> 
>>> Sergi,
>>> 
>>> Thanks, now I found the exact conditions that switch the indexes from
>>> on-heap to off-heap.
>>> 
>>> But looks like you did a misprint in your conclusion saying that
>>> setOffHeapMaxMemory should be 0.
>>>> Thus from the user standpoint we can say, that if either
>>> setOffHeapMaxMemory
>>>>> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will
>>> go
>>>> off-heap.
>>> 
>>> In fact, as you elaborated earlier, it has to be >= 0 which leads to the
>>> following final statement:
>>> 
>>> SQL engine will store indexes in the off-heap region if
>>> setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported
>>> off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0
>>> automatically if it’s value is less than -1 and one of off-heap memory
>>> modes is used.
>>> 
>>> Are we on the same page now?
>>> 
>>> —
>>> Denis
>>> 
>>>> On Nov 4, 2016, at 1:51 AM, Sergi Vladykin <sergi.vladykin@gmail.com <mailto:sergi.vladykin@gmail.com>>
>>> wrote:
>>>> 
>>>> Denis,
>>>> 
>>>> I checked the code now as well. I actually was wrong, but your statement
>>> is
>>>> wrong either.
>>>> 
>>>> If we are talking about indexes then the exact condition when index will
>>> be
>>>> off-heap is the following: either setOffHeapMaxMemory >= 0 or
>>> setMemoryMode
>>>> is OFFHEAP_TIERED.
>>>> 
>>>> But if setMemoryMode is set to one of off-heap modes and
>>> setOffHeapMaxMemory is
>>>> -1, then setOffHeapMaxMemory is silently getting switched to 0.
>>>> 
>>>> Thus from the user standpoint we can say, that if either
>>> setOffHeapMaxMemory
>>>>> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will
>>> go
>>>> off-heap.
>>>> 
>>>> Sergi
>>>> 
>>>> 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org <mailto:dsetrakyan@apache.org>>:
>>>> 
>>>>> Do we currently throw an exception if memory mode is set to off-heap
by
>>> the
>>>>> max-offheap-memory is set to "-1"? Otherwise, it can get very confusing
>>> to
>>>>> users.
>>>>> 
>>>>> On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <dmagda@gridgain.com
<mailto:dmagda@gridgain.com>>
>>> wrote:
>>>>> 
>>>>>> Sergi,
>>>>>> 
>>>>>> Referring to your response and the source code I should say that,
in
>>>>> fact,
>>>>>> the documentation has to state the following:
>>>>>> 
>>>>>> “To place both data and indexes in off heap memory CacheConfiguration.
>>>>> setMemoryMode
>>>>>> has to be set to one of off heap modes and CacheConfiguration.
>>>>> setOffHeapMaxMemory
>>>>>> has to be set to a value different from -1 (disabled). If at least
one
>>> of
>>>>>> these conditions is not met then both data and indexes will be located
>>> in
>>>>>> on heap memory”
>>>>>> 
>>>>>> Is this statement correct?
>>>>>> 
>>>>>> —
>>>>>> Denis
>>>>>> 
>>>>>>> On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <sergi.vladykin@gmail.com
<mailto:sergi.vladykin@gmail.com>>
>>>>>> wrote:
>>>>>>> 
>>>>>>> Prachi,
>>>>>>> 
>>>>>>> In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory
>>>>>>> property, so [1] is correct. If setOffHeapMaxMemory is disabled
(set
>>> to
>>>>>>> -1), then indexes will never be stored offheap, regardless of
>>>>>> setMemoryMode
>>>>>>> setting.
>>>>>>> 
>>>>>>> Probably we have to change [2] to make it clear, that it is about
>>>>>>> setOffHeapMaxMemory
>>>>>>> but not about setMemoryMode.
>>>>>>> 
>>>>>>> Sergi
>>>>>>> 
>>>>>>> 
>>>>>>> 2016-11-03 2:25 GMT+03:00 Prachi Garg <pgarg@gridgain.com
<mailto:pgarg@gridgain.com>>:
>>>>>>> 
>>>>>>>> Engineers,
>>>>>>>> 
>>>>>>>> I was going through documentation for Off-Heap SQL Indexes
>>>>>>>> <https://apacheignite.readme.io/docs/sql-queries#off-heap-
<https://apacheignite.readme.io/docs/sql-queries#off-heap->
>>> sql-indexes
>>>>>> 
>>>>>> and
>>>>>>>> found two contradicting statements-
>>>>>>>> 
>>>>>>>> According to [1] -  " `CacheConfiguration.setOffHeapMaxMemory`
is
>>> the
>>>>>> only
>>>>>>>> property to enable or disable off-heap indexing. While
>>>>>>>> CacheConfiguration.setMemoryMode is used to configure off-heap
>>> memory
>>>>>>>> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory
<https://apacheignite.readme.io/v1.7/docs/off-heap-memory>>, it does
>>>>> not
>>>>>>>> have any effect on indexing."
>>>>>>>> 
>>>>>>>> According to [2] -  "Note that when off-heap memory is configured,
>>>>>> Ignite
>>>>>>>> will store query indexes off-heap as well."
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Which one is true?
>>>>>>>> 
>>>>>>>> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap-
<https://apacheignite.readme.io/docs/sql-queries#off-heap->
>>>>>> sql-indexes
>>>>>>>> 
>>>>>>>> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory
<http://apacheignite.gridgain.org/docs/off-heap-memory>
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> 
>>>>>>>> -Prachi
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>> 
>>> 
> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message