ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: [DISCUSSION] Urgent Ignite bug fix release
Date Mon, 28 Aug 2017 11:38:07 GMT
Looks good, but why in the example provided are we suggesting 8GB? 2 nodes
with 8GB will completely exhaust the available memory. I would suggest 6 or
7GB.

Also, why 100MB for default policy. Anything under 1GB seems too small.

Can you please comment?

D.

On Mon, Aug 28, 2017 at 3:31 AM, Seliverstov Igor <gvvinblade@gmail.com>
wrote:

> One more example of possible warning:
>
> -----------------------------------------------------
> Excessive memory usage by Ignite node process (performance may drop)
> [requested=44613MB, available=15942MB].
>
> Please tune the folowing settings as suggested:
>   MemoryPolicyConfiguration.initialSize for bigPlc: 8102MB
>   MemoryPolicyConfiguration.maxSize     for bigPlc: 8102MB
>   MemoryPolicyConfiguration.initialSize for dfltPlc: 100MB
>   MemoryPolicyConfiguration.maxSize     for dfltPlc: 100MB
>
> Current settings:
>   Java Heap  maxSize: 3543MB
>   Java Heap initSize: 250MB
>   MemoryPolicyConfiguration.initialSize for bigPlc: 256MB
>   MemoryPolicyConfiguration.maxSize     for bigPlc: 40960MB
>   MemoryPolicyConfiguration.initialSize for dfltPlc: 10MB
>   MemoryPolicyConfiguration.maxSize     for dfltPlc: 10MB
>   The overall expected memory usage by all Ignite nodes on the host:
> 44613MB
> -----------------------------------------------------
>
> Your thoughts?
>
> 2017-08-28 5:06 GMT+03:00 Denis Magda <dmagda@apache.org>:
>
> > Guys,
> >
> > ML lib profile is missing in 2.1 release! That must be fixed and rolled
> > out in this emergency release:
> > https://issues.apache.org/jira/browse/IGNITE-6193 <
> > https://issues.apache.org/jira/browse/IGNITE-6193>
> >
> > Oleg, Yuri, please step in and handle the issue.
> >
> > BTW, who is considered to be the release manager of this release?
> >
> > —
> > Denis
> >
> > > On Aug 25, 2017, at 2:29 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> > wrote:
> > >
> > > I like the format proposed by Denis, very clear.
> > >
> > > However, I also do not understand why a user should change the size of
> > some
> > > system cache. How would a user ever know what value to put there? This
> > > value should be configured by Ignite automatically.
> > >
> > > D.
> > >
> > > On Fri, Aug 25, 2017 at 2:24 PM, Denis Magda <dmagda@apache.org>
> wrote:
> > >
> > >> Igor,
> > >>
> > >> Let me suggest this format.
> > >>
> > >> ---------------------------------------------
> > >> Excessive memory usage by Ignite node process (performance may drop)
> > >> [requested=29251MB, available=15942MB]
> > >>
> > >> Please tune the following settings:
> > >>  [MemoryConfiguration.defaultMemoryPolicySize = suggested value]
> > >>  MemoryConfiguration.systemCacheMaxSize = suggested value
> > >>  [MemoryPolicyConfiguration.maxSize for {policy_name_1} = suggested
> > >> value]
> > >>  [MemoryPolicyConfiguration.maxSize for {policy_name_2} = suggested
> > >> value]
> > >>
> > >> Current settings:
> > >>   [DefaultMemoryPolicySize = value]
> > >>   [{policy_name_1} size = value]
> > >>   [{policy_name_1} size = value]
> > >>   SystemCacheInitialSize = value
> > >>   SystemCacheMaxSize = value
> > >>   Java Heap Init Size = value
> > >>   Java Heap Max Size = value
> > >>
> > >> The overall memory usage by all Ignite nodes on the host: value
> > >> -------------------------------------------
> > >>
> > >> Records in […] are optional. If custom memory policy is not set or the
> > >> default memory policy is overridden the output will miss some of the
> > rows.
> > >>
> > >> As for systemCacheMaxSize, it should be show ONLY if the parameter was
> > set
> > >> explicitly by user code. Otherwise, the platform should be wise enough
> > to
> > >> instantiate it properly depending on the host memory usage.
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Aug 25, 2017, at 1:49 PM, Seliverstov Igor <gvvinblade@gmail.com>
> > >> wrote:
> > >>>
> > >>> The message without logging layout:
> > >>>
> > >>> Not enough memory for current process [required=29251MB,
> > >> available=15942MB].
> > >>> Please change MemoryConfiguration.systemCacheMaxSize and
> > >>> MemoryConfiguration.defaultMemoryPolicySize to decrease memory
> > allocated
> > >>> for each node.
> > >>>
> > >>> Current settings:
> > >>>  HeapInit=250MB
> > >>>  HeapMax=3543MB
> > >>>  DefaultMemoryPolicySize=12753MB
> > >>>  SystemCacheInitialSize=40MB
> > >>>  SystemCacheMaxSize=100MB
> > >>>
> > >>> Other ignite instanses on the server require: 12853MB
> > >>>
> > >>> I think it's make sense to describe what these numbers are consist
> of.
> > >>>
> > >>> We simple say which parameters have an impact on how much memory the
> > >>> instance needs and their (parameters) actual values.
> > >>>
> > >>> Also we notice that more than one Ignite instance are ran on the
> server
> > >> or
> > >>> workstation and it also consumes memory.
> > >>>
> > >>> 25 авг. 2017 г. 21:30 пользователь "Dmitriy Setrakyan"
<
> > >>> dsetrakyan@apache.org> написал:
> > >>>
> > >>>> Igor, what is this flood of WARN messaging coming after the text?
> Are
> > we
> > >>>> really going to print this whole thing out?
> > >>>>
> > >>>> On Fri, Aug 25, 2017 at 9:49 AM, Seliverstov Igor <
> > gvvinblade@gmail.com
> > >>>
> > >>>> wrote:
> > >>>>
> > >>>>> This message appears on topology change in case the available
> memory
> > is
> > >>>>> exceeded
> > >>>>>
> > >>>>> 2017-08-25 19:47 GMT+03:00 Seliverstov Igor <gvvinblade@gmail.com
> >:
> > >>>>>
> > >>>>>> An example of current impl:
> > >>>>>>
> > >>>>>>
> > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Not enough
memory
> for
> > >>>>>> current process [required=29251MB, available=15942MB].
> > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Please change
> > >>>>>> MemoryConfiguration.systemCacheMaxSize and MemoryConfiguration.
> > >>>>> defaultMemoryPolicySize
> > >>>>>> to decrease memory allocated for each node.
> > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Current settings:
> > >>>>>> [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]   HeapInit=250MB
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]   HeapMax=3543MB
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>> DefaultMemoryPolicySize=
> > >>>>>> 12753MB
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>>>> SystemCacheInitialSize=40MB
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>> SystemCacheMaxSize=100MB
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager] Other ignite
> > instanses
> > >>>> on
> > >>>>>> the server require: 12853MB
> > >>>>>> [2017-08-25 19:44:37,741][WARN ][disco-event-worker-#29%internal.
> > >>>>>> GridHomePathSelfTest0%][GridDiscoveryManager]
> > >>>>>>
> > >>>>>>
> > >>>>>> 2017-08-25 17:40 GMT+03:00 Sergey Kozlov <skozlov@gridgain.com>:
> > >>>>>>
> > >>>>>>> I suppose we should not forget JVM heap size and suggest
to
> reduce
> > >>>> both
> > >>>>>>> options
> > >>>>>>>
> > >>>>>>> On Fri, Aug 25, 2017 at 5:24 PM, Dmitriy Setrakyan
<
> > >>>>> dsetrakyan@apache.org
> > >>>>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Igor, I would change the message. How about this:
> > >>>>>>>>
> > >>>>>>>> Required RAM size is larger than total physical
memory available
> > for
> > >>>>> OS.
> > >>>>>>>>> Please change MemoryConfiguration.WhichProperty
and
> > >>>>>>>>> MemoryPolicyConfiguration.WhichProperty to
decrease memory
> > >>>>> allocated
> > >>>>>>> for
> > >>>>>>>>> each node.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Also, can we calculate what the memory size allocated
for each
> > node
> > >>>>>>> should
> > >>>>>>>> be? In that case we should suggest it.
> > >>>>>>>>
> > >>>>>>>> D.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Aug 25, 2017 at 7:20 AM, Seliverstov Igor
<
> > >>>>> gvvinblade@gmail.com
> > >>>>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> What do you, guys think about next warning?
> > >>>>>>>>>
> > >>>>>>>>> [2017-08-25 17:17:04,718][INFO
> > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat
> > >>>>>>> hSelfTest0]
> > >>>>>>>>> System cache's MemoryPolicy size is configured
to 40 MB. Use
> > >>>>>>>>> MemoryConfiguration.systemCacheMemorySize property
to change
> the
> > >>>>>>>> setting.
> > >>>>>>>>> [2017-08-25 17:17:04,718][WARN
> > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat
> > >>>>>>> hSelfTest0]
> > >>>>>>>>>
> > >>>>>>>>>>>> Required RAM size is larger than
total physical memory
> > >>>> available
> > >>>>>>> for
> > >>>>>>>>> OS.
> > >>>>>>>>>>>> Check your configuration to avoid
swap partition usage.
> > >>>>>>>>>>>> Use MemoryConfiguration and MemoryPolicyConfiguration
to
> > >>>> change
> > >>>>>>> the
> > >>>>>>>>> settings.
> > >>>>>>>>>>>> Physical memory [required=16397MB,
available=15942MB]
> > >>>>>>>>>
> > >>>>>>>>> [2017-08-25 17:17:04,726][WARN
> > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat
> > >>>>>>> hSelfTest0]
> > >>>>>>>>> Peer class loading is enabled (disable it in
production for
> > >>>>>>> performance
> > >>>>>>>> and
> > >>>>>>>>> deployment consistency reasons)
> > >>>>>>>>> [2017-08-25 17:17:04,726][INFO
> > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat
> > >>>>>>> hSelfTest0]
> > >>>>>>>>> Configured caches [in 'sysMemPlc' memoryPolicy:
> > >>>>> ['ignite-sys-cache']]
> > >>>>>>>>> [2017-08-25 17:17:04,731][INFO
> > >>>>>>>>> ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePat
> > >>>>>>> hSelfTest0]
> > >>>>>>>>> 3-rd party licenses can be found at:
> > >>>>>>>>> /home/gvvinblade/projects/ignite/incubator-ignite/libs/
> licenses
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> 2017-08-25 13:26 GMT+03:00 Yakov Zhdanov <yzhdanov@apache.org
> >:
> > >>>>>>>>>
> > >>>>>>>>>> Agree, let's release new version including
tickets mentioned
> by
> > >>>>>>> Denis
> > >>>>>>>> and
> > >>>>>>>>>> Ivan.
> > >>>>>>>>>>
> > >>>>>>>>>> --Yakov
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Sergey Kozlov
> > >>>>>>> GridGain Systems
> > >>>>>>> www.gridgain.com
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message