ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stanislav Lukyanov <stanlukya...@gmail.com>
Subject RE: Runtime.availableProcessors() returns hardware's CPU count whichis the issue with Ignite in Kubernetes
Date Tue, 26 Dec 2017 13:30:38 GMT
Hi Arseny,

This behavior of the `Runtime.availableProcessors()` is actually a recognized issue of the
Hotspot, see [1]. It was fixed not that long ago for JDK 9 and 8uX, and I can see correct
values returned in my Docker environment with JDK 8u151, although I believe it depends on
a specific way a container is configured.

Which Java version do you use? Can you try your code on JDK 8u151?

BTW see also [2] and [3] on more stuff to be fixed in JDK for better container support.

Thanks,
Stan

[1] https://bugs.openjdk.java.net/browse/JDK-6515172
[2] https://bugs.openjdk.java.net/browse/JDK-8146115
[3] https://bugs.openjdk.java.net/browse/JDK-8182070


From: Yakov Zhdanov
Sent: 26 декабря 2017 г. 16:05
To: user@ignite.apache.org; dev@ignite.apache.org
Subject: Re: Runtime.availableProcessors() returns hardware's CPU count whichis the issue
with Ignite in Kubernetes

Cross-posting to dev list.

Guys,

Suggestion below makes sense to me. Filed a ticket
https://issues.apache.org/jira/browse/IGNITE-7310

Perhaps, Arseny would like to provide a PR himself ;)

--Yakov

2017-12-26 14:32 GMT+03:00 Arseny Kovalchuk <arseny.kovalchuk@synesis.ru>:

> Hi guys.
>
> Ignite configures all thread pools, selectors, etc. basing on Runtime.availableProcessors()
> which seems not correct in containerized environment. In Kubernetes with
> Docker that method returns CPU count of a Node/machine, which is 64 in our
> particular case. But those 64 CPU and their timings are shared between
> other stuff on the node like other Pods and services. Appropriate value of
> available cores for Pod is usually configured as CPU Resource and estimated
> basing on different things taking performance into account. General idea,
> if you want to run several Pods on the same node, they all should request
> less resources then the node provides. So, we give 4-8 cores for Ignite
> instance in Kubernetes, but Ignite's thread pools are configured like they
> get all 64 CPUs, and in turn we get a lot of threads for the Pod with 4-8
> cores available.
>
> Now we manually set appropriate values for all available properties which
> relate to thread pools.
>
> Would it be correct to have one environment variable, say
> IGNITE_CONCURRENCY_LEVEL which will be used as a reference value for those
> configurations and by default equals to Runtime.availableProcessors()?
>
> Thanks.
>
> ​
> Arseny Kovalchuk
>
> Senior Software Engineer at Synesis
> skype: arseny.kovalchuk
> mobile: +375 (29) 666-16-16
> ​LinkedIn Profile <http://www.linkedin.com/in/arsenykovalchuk/en>​
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message