ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ilya Kasnacheev <ilya.kasnach...@gmail.com>
Subject Re: Impossible memory region created in TC test
Date Mon, 15 Apr 2019 07:18:28 GMT
Hello!

Ignite does not touch every page of data region, and until you touch a page
OS will not allocate any specific physical RAM to the virtual RAM address
of that page.

Moreover, AFAIK Ignite will not even allocate all the memory permitted by
data region until it is needed. It will allocate memory in chunks, which
means your system will slow to grind trying to find RAM for the next chunk
as you try to load data into such node.

If you would use Pre-Touch feature which was suggested in this developer
list a few months ago, you will see it fail explicitly.

Regards,
-- 
Ilya Kasnacheev


пн, 15 апр. 2019 г. в 09:49, Nikolay Izhikov <nizhikov@apache.org>:

> Hello, Igniters.
>
> Why test on TC can allocate Data Region bigger than the amount of
> available RAM?
> How can I disable this feature in a specific test?
>
> "memoryAllocated=1024.0 GiB ... required=1093413MB, available=96675MB"
>
> I have a test [1] that try to create Data Region of 1024 Gib size.
> I want to check OOM exception that should occur while this DataRegion
> starting.
>
> Locally, I have it.
>
> But, On TC that DataRegion successfully created [2]
> I see following log messages:
>
> ```
> [2019-04-13 21:31:30,426][INFO
> ][exchange-worker-#91818%test-server%][PageMemoryImpl] Started page memory
> [memoryAllocated=1024.0 GiB, pages=260204016, tableSize=19.8 GiB,
> checkpointBuffer=2.0 GiB]
> ....
> [2019-04-13 21:31:18,004][WARN
> ][test-runner-#91737%pagemem.PageMemoryLazyAllocationWithPDSTest%][IgniteKernal%test-server]
> Nodes started on local machine require more than 80% of physical RAM what
> can lead to significant slowdown due to swapping (please decrease JVM heap
> size, data region size or checkpoint buffer size) [required=1093413MB,
> available=96675MB]
> ```
>
> [1]
> https://github.com/apache/ignite/pull/6388/files#diff-754f5ea2f3579b81c50662f3d20c9126R138
>
> [2]
> https://ci.ignite.apache.org/viewLog.html?buildId=3591587&buildTypeId=IgniteTests24Java8_RunAll#testNameId-7083016202406489951
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message