asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianfeng Jia <jianfeng....@gmail.com>
Subject Re: How to set the lsm component size?
Date Mon, 12 Sep 2016 19:29:06 GMT
I was a little confused, there is another configuration:

storage.memorycomponent.globalbudget ( which I set to 4G) 

I was thinking this is the budget that every component on one partition is shared. Is that
the case?

> On Sep 12, 2016, at 12:16 PM, Sattam Alsubaiee <salsubaiee@gmail.com> wrote:
> 
> The 128M is shared by all the memory components of the primary index and
> all its secondary indexes across all io devices on that node.
> Also the in-memory components usually usually has fill factor of 75% since
> the pages are 75% full and the remaining 25% is un-utilized.
> 
> The page size that you have set 128KB looks reasonable for most cases. Your
> best bet is to increase the value of storage.memorycomponent.numpage to a
> higher number.
> 
> Sattam
> 
> 
> On Mon, Sep 12, 2016 at 11:33 AM, Jianfeng Jia <jianfeng.jia@gmail.com>
> wrote:
> 
>> Dear devs,
>> 
>> I’m using the `no-merge` compaction policy and find that the physical
>> flushed on-disk component is smaller than I was expected.
>> 
>> Here are my related configurations
>> 
>>  <property>
>>    <name>storage.memorycomponent.pagesize</name>
>>    <value>128KB</value>
>>    <description>The page size in bytes for pages allocated to memory
>>      components. (Default = "131072" // 128KB)
>>    </description>
>>  </property>
>> 
>>  <property>
>>    <name>storage.memorycomponent.numpages</name>
>>    <value>1024</value>
>>    <description>The number of pages to allocate for a memory component.
>>      (Default = 256)
>>    </description>
>>  </property>
>> 
>> With these two settings, I’m expecting the lsm component should be 128M.
>> However, the flushed one is about 16M~ 20M. Do we have some compression for
>> the on-disk components? If so, it will be good. Otherwise, could someone
>> help me to increase the component size? Thanks!
>> 
>> Best,
>> 
>> Jianfeng Jia
>> PhD Candidate of Computer Science
>> University of California, Irvine
>> 
>> 



Best,

Jianfeng Jia
PhD Candidate of Computer Science
University of California, Irvine


Mime
View raw message