Mailing-List: contact user-help@kudu.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@kudu.apache.org
MIME-Version: 1.0
From: Jason Heo <jason.heo.sde@gmail.com>
Date: Tue, 25 Apr 2017 00:24:59 +0900
Message-ID: <CAO3rhrSqGk5HkEJiO9M3hHQYOfwC4K2GaDZE6XKAPp8GCjsgPw@mail.gmail.com>
Subject: Question about memory_limit_hard_bytes and
To: user@kudu.apache.org
Content-Type: multipart/alternative; boundary=001a11c029b2de703a054deb35a3
archived-at: Mon, 24 Apr 2017 15:25:04 -0000

--001a11c029b2de703a054deb35a3
Content-Type: text/plain; charset=UTF-8

Hello.

I'm using Apache Kudu 1.3.0.

I found some explanation in manual and cdh about memory_limit_hard and
block_cache_capacity But I couldn't understand what are they exactly used
for, what is stored at block_cache, and what's the meaning of hard limit of
memory?

In Kudu Configuration Reference:

   - *memory_limit_hard_bytes
   <https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_memory_limit_hard_bytes>*:
   Maximum amount of memory this daemon should use, in bytes. A value of 0
   autosizes based on the total system memory. A value of -1 disables all
   memory limiting.
   - *block_cache_capacity_mb
   <https://kudu.apache.org/docs/configuration_reference.html#kudu-tserver_block_cache_capacity_mb>*:
   block cache capacity in MB


In Cloudera Manager's help:

   - *memory_limit_hard_bytes*: Maximum amount of memory that the Kudu
   Tablet Server will use before it starts rejecting all incoming writes.
   - *block_cache_capacity_mb*: Maximum amount of memory allocated to the
   Kudu Tablet Server's block cache.

Is hard_limit related with insert/update/delete and is not related to
performance of read? Does block_cache store tablet data? Then what's the
relationship with page cache.

After watching Comcast presentation, I was surprised because they are using
memory_limit_hard=4G, block_cache=512MB.

Regards,

Jason

--001a11c029b2de703a054deb35a3
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello.<div><br></div><div>I&#39;m using Apache Kudu 1.3.0.=
</div><div><br></div><div>I found some explanation in manual and cdh about =
memory_limit_hard and block_cache_capacity But I couldn&#39;t understand wh=
at are they exactly used for, what is stored at block_cache, and what&#39;s=
 the meaning of hard limit of memory?</div><div><div><br></div><div>In Kudu=
 Configuration Reference:</div><div><ul><li><b style=3D"font-weight:bold"><=
a href=3D"https://kudu.apache.org/docs/configuration_reference.html#kudu-ts=
erver_memory_limit_hard_bytes">memory_limit_hard_bytes</a></b>: Maximum amo=
unt of memory this daemon should use, in bytes. A value of 0 autosizes base=
d on the total system memory. A value of -1 disables all memory limiting.<b=
r></li><li><b><a href=3D"https://kudu.apache.org/docs/configuration_referen=
ce.html#kudu-tserver_block_cache_capacity_mb">block_cache_capacity_mb</a></=
b>: block cache capacity in MB<br></li></ul></div><div><br></div><div>In Cl=
oudera Manager&#39;s help:</div><div><ul><li><b>memory_limit_hard_bytes</b>=
:=C2=A0Maximum amount of memory that the Kudu Tablet Server will use before=
 it starts rejecting all incoming writes.<br></li><li><b>block_cache_capaci=
ty_mb</b>:=C2=A0Maximum amount of memory allocated to the Kudu Tablet Serve=
r&#39;s block cache.<br></li></ul><div>Is hard_limit related with insert/up=
date/delete and is not related to performance of read? Does block_cache sto=
re tablet data? Then what&#39;s the relationship with page cache.</div></di=
v></div><div><br></div><div>After watching Comcast presentation, I was surp=
rised because they are using memory_limit_hard=3D4G, block_cache=3D512MB.</=
div><div><br></div><div>Regards,</div><div><br></div><div>Jason</div></div>

--001a11c029b2de703a054deb35a3--