Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <1512110706389-0.post@n6.nabble.com>
References: <1512110706389-0.post@n6.nabble.com>
From: Alexey Kukushkin <kukushkinalexey@gmail.com>
Date: Fri, 1 Dec 2017 16:42:47 +0300
Message-ID: <CA+vFNSrwnAeFGtXFMzxS_-ige-7a6VsacnOhGNQcBCVQN2Xb_Q@mail.gmail.com>
Subject: Re: Out of memory error
To: user@ignite.apache.org
Content-Type: multipart/alternative; boundary="94eb2c192b20534743055f478b87"
archived-at: Fri, 01 Dec 2017 13:42:53 -0000

--94eb2c192b20534743055f478b87
Content-Type: text/plain; charset="UTF-8"

Hi,

You identified the problem right: there is not enough memory to handle 15G
of Postgres data on your server. Your idea to configure a memory policy to
increase available memory is right but 16G is also not enough. Ignite data
size is noticeably larger (up to 3 times, depends on many factors like
amount of indexes) than that of the relational DB data files on disk.

You need to estimate how much memory you need. There are two options:

   - Theoretical - see here
   <https://apacheignite.readme.io/docs/capacity-planning>.
   - Empirical: enable memory metrics
   <https://apacheignite.readme.io/docs/memory-metrics> and load a smaller
   amount of entries into cache. Use JMX client to check "totalAllocatedPages"
   of the DataRegionMetricsMXBean on the server. Multiply the number by 4K to
   get total used memory in kilobytes. Then extrapolate to the max amount of
   entries you are going to have.

Remember that Ignite saves you money by horizontal scaling. Just have 4 32G
servers if you need 100G of RAM.

--94eb2c192b20534743055f478b87
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra">Hi,
</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">You i=
dentified the problem right: there is not enough memory to handle 15G of Po=
stgres data on your server. Your idea to configure a memory policy to incre=
ase available memory is right but 16G is also not enough. Ignite data size =
is noticeably larger (up to 3 times, depends on many factors like amount of=
 indexes) than that of the relational DB data files on disk.</div><div clas=
s=3D"gmail_extra"><br></div><div class=3D"gmail_extra">You need to estimate=
 how much memory you need. There are two options:</div><div class=3D"gmail_=
extra"><ul><li><a href=3D"https://apacheignite.readme.io/docs/capacity-plan=
ning">Theoretical - see here</a>.</li><li>Empirical: <a href=3D"https://apa=
cheignite.readme.io/docs/memory-metrics">enable memory metrics</a> and load=
 a smaller amount of entries into cache. Use JMX client to check &quot;tota=
lAllocatedPages&quot; of the DataRegionMetricsMXBean on the server. Multipl=
y the number by 4K to get total used memory in kilobytes. Then extrapolate =
to the max amount of entries you are going to have.</li></ul><div>Remember =
that Ignite saves you money by horizontal scaling. Just have 4 32G servers =
if you need 100G of RAM.=C2=A0</div></div><div class=3D"gmail_extra">=C2=A0=
</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra"><br><=
/div></div>

--94eb2c192b20534743055f478b87--