Mailing-List: contact dev-help@asterixdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@asterixdb.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Subject: Re: How to set the lsm component size?
From: Jianfeng Jia <jianfeng.jia@gmail.com>
In-Reply-To: <CAEYY6db4UGoRd8N3fmiCi8uPWwyyZSEFn-s6n-vRPkJDtvPfNQ@mail.gmail.com>
Date: Mon, 12 Sep 2016 12:29:06 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <8FA6BB7D-4D55-4D6C-A463-9CCDFD9F28D7@gmail.com>
References: <5F77077E-6828-4FFB-AAD4-F467680DB8B6@gmail.com> <CAEYY6db4UGoRd8N3fmiCi8uPWwyyZSEFn-s6n-vRPkJDtvPfNQ@mail.gmail.com>
To: dev@asterixdb.apache.org
archived-at: Mon, 12 Sep 2016 19:29:22 -0000

I was a little confused, there is another configuration:

storage.memorycomponent.globalbudget ( which I set to 4G)=20

I was thinking this is the budget that every component on one partition =
is shared. Is that the case?

> On Sep 12, 2016, at 12:16 PM, Sattam Alsubaiee <salsubaiee@gmail.com> =
wrote:
>=20
> The 128M is shared by all the memory components of the primary index =
and
> all its secondary indexes across all io devices on that node.
> Also the in-memory components usually usually has fill factor of 75% =
since
> the pages are 75% full and the remaining 25% is un-utilized.
>=20
> The page size that you have set 128KB looks reasonable for most cases. =
Your
> best bet is to increase the value of storage.memorycomponent.numpage =
to a
> higher number.
>=20
> Sattam
>=20
>=20
> On Mon, Sep 12, 2016 at 11:33 AM, Jianfeng Jia =
<jianfeng.jia@gmail.com>
> wrote:
>=20
>> Dear devs,
>>=20
>> I=E2=80=99m using the `no-merge` compaction policy and find that the =
physical
>> flushed on-disk component is smaller than I was expected.
>>=20
>> Here are my related configurations
>>=20
>>  <property>
>>    <name>storage.memorycomponent.pagesize</name>
>>    <value>128KB</value>
>>    <description>The page size in bytes for pages allocated to memory
>>      components. (Default =3D "131072" // 128KB)
>>    </description>
>>  </property>
>>=20
>>  <property>
>>    <name>storage.memorycomponent.numpages</name>
>>    <value>1024</value>
>>    <description>The number of pages to allocate for a memory =
component.
>>      (Default =3D 256)
>>    </description>
>>  </property>
>>=20
>> With these two settings, I=E2=80=99m expecting the lsm component =
should be 128M.
>> However, the flushed one is about 16M~ 20M. Do we have some =
compression for
>> the on-disk components? If so, it will be good. Otherwise, could =
someone
>> help me to increase the component size? Thanks!
>>=20
>> Best,
>>=20
>> Jianfeng Jia
>> PhD Candidate of Computer Science
>> University of California, Irvine
>>=20
>>=20


Best,

Jianfeng Jia
PhD Candidate of Computer Science
University of California, Irvine