Mailing-List: contact user-help@kudu.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@kudu.apache.org
MIME-Version: 1.0
In-Reply-To: <1459AEE0-965D-47F5-9318-2C788407BD7F@onfocus.io>
References: <1459AEE0-965D-47F5-9318-2C788407BD7F@onfocus.io>
From: Paul Brannan <paul.brannan@thesystech.com>
Date: Tue, 28 Feb 2017 11:22:51 -0500
Message-ID: <CAEeV1ntTGDam1eg8qwYJCM9UdJ+7xbiwksQF2HksdoOA+xjpeg@mail.gmail.com>
Subject: Re: Recommended bulk size ?
To: user@kudu.apache.org
Content-Type: multipart/alternative; boundary=001a11458eb28be4660549999ba5
archived-at: Tue, 28 Feb 2017 16:22:58 -0000

--001a11458eb28be4660549999ba5
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

As you said, I expect it depends on many variables.  I ran a quick & dirty
experiment when first evaluating kudu 1.0 to see how flushing at varying
intervals affected insert rates.  I had one master and one tserver, each in
the default configuration, on an ext4 filesystem on a spinning disk.  The
table had two string columns "key" and "value", both part of the primary
key, each less than 30 bytes.  Here were the results:

Manual flush every insert: 100K inserts in 14.5s (~7K/s)
Manual flush every 100K: 1M inserts in 4.7s (~215K/s, w/ warnings about
"blocked reactor thread")
Manual flush every 10K: 1M inserts in 4.2s (~240K/s)
Auto flush background, no explicit flush: 1M inserts in 4.8s (w/ warnings
about "blocked reactor thread" and "thread stuck")
Auto flush background, explicit flush every 10K inserts: 1M inserts in 4.2s
(~240K/s)
Async flush every 10K inserts: 1M inserts in 2.8s (~350K/s)
Async flush every 1K inserts: 1M inserts in 2.7s (~370K/s)
Async flush every 100: 1M inserts in 3.3s (~300K/s)
Async flush every 10: 1M inserts in 10.6s (~95K/s)

Based on this experiment, I chose async flush with a 1K interval, because
beyond that there is diminishing return, and I don't want to run out of
mutation space.


On Tue, Feb 28, 2017 at 6:29 AM, Nicolas Fouch=C3=A9 <nfouche@onfocus.io> w=
rote:

> Hi. Is there any recommendation on the number of operations in
> bulk/AUTO_FLUSH_BACKGROUND ? I guess it highly depends on the cluster siz=
e,
> the number of partitions hit by the operations, etc. But there could be
> some guidelines out there ?
>
>
> Looking at the code of the kudu client, it seems that the default size is
> 1000: `private int mutationBufferSpace =3D 1000;`.
>
> - Nicolas
>

--001a11458eb28be4660549999ba5
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>As you said, I expect it depends on many variables.=
=C2=A0 I ran a quick &amp; dirty experiment when first evaluating kudu 1.0 =
to see how flushing at varying intervals affected insert rates.=C2=A0 I had=
 one master and one tserver, each in the default configuration, on an ext4 =
filesystem on a spinning disk.=C2=A0 The table had two string columns &quot=
;key&quot; and &quot;value&quot;, both part of the primary key, each less t=
han 30 bytes.=C2=A0 Here were the results:<br></div><div><br></div><div>Man=
ual flush every insert: 100K inserts in 14.5s (~7K/s)<br></div><div>Manual =
flush every 100K: 1M inserts in 4.7s (~215K/s, w/ warnings about &quot;bloc=
ked reactor thread&quot;)<br></div><div>Manual flush every 10K: 1M inserts =
in 4.2s (~240K/s)<br></div><div>Auto flush background, no explicit flush: 1=
M inserts in 4.8s (w/ warnings about &quot;blocked reactor thread&quot; and=
 &quot;thread stuck&quot;)<br></div><div>Auto flush background, explicit fl=
ush every 10K inserts: 1M inserts in 4.2s (~240K/s)<br></div><div>Async flu=
sh every 10K inserts: 1M inserts in 2.8s (~350K/s)<br></div><div>Async flus=
h every 1K inserts: 1M inserts in 2.7s (~370K/s)<br></div><div>Async flush =
every 100: 1M inserts in 3.3s (~300K/s)<br></div><div>Async flush every 10:=
 1M inserts in 10.6s (~95K/s)<br><br></div><div>Based on this experiment, I=
 chose async flush with a 1K interval, because beyond that there is diminis=
hing return, and I don&#39;t want to run out of mutation space.<br></div><d=
iv><br></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote=
">On Tue, Feb 28, 2017 at 6:29 AM, Nicolas Fouch=C3=A9 <span dir=3D"ltr">&l=
t;<a href=3D"mailto:nfouche@onfocus.io" target=3D"_blank">nfouche@onfocus.i=
o</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u>


<div>
<div>
<p style=3D"margin:0px;font-size:12px;line-height:normal">Hi. Is there any =
recommendation on the number of operations in bulk/AUTO_FLUSH_BACKGROUND ? =
I guess it highly depends on the cluster size, the number of partitions hit=
 by the operations, etc. But there could be some guidelines out there ?</p>
<p style=3D"margin:0px;font-size:12px;line-height:normal"><br></p>
<p style=3D"margin:0px;font-size:12px;line-height:normal">Looking at the co=
de of the kudu client, it seems that the default size is 1000: `private int=
 mutationBufferSpace =3D 1000;`.</p><span class=3D"HOEnZb"><font color=3D"#=
888888">
</font></span></div><span class=3D"HOEnZb"><font color=3D"#888888">
<div><br></div>
<div>- Nicolas</div>
<div class=3D"m_1429966963747156059unibox-signature">

</div>
</font></span></div>
</blockquote></div><br></div>

--001a11458eb28be4660549999ba5--