ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: [IGNITE-5717] improvements of MemoryPolicy default size
Date Wed, 02 Aug 2017 07:37:09 GMT
On Wed, Aug 2, 2017 at 9:33 AM, Vladimir Ozerov <vozerov@gridgain.com>
wrote:

> Dima,
>
> Probably folks who worked closely with storage know why.
>

Without knowing why, how can we make a decision?

Alexey Goncharuk, was it you who made the decision about not using
increments? Do know remember what was the reason?


>
> The very problem is that before being started once on production
> environment, Ignite will typically be started hundred times on developer's
> environment. I think that default should be ~10% of total RAM.
>

Why not 80% of *free *RAM?


>
> On Wed, Aug 2, 2017 at 10:21 AM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
>
> > On Wed, Aug 2, 2017 at 7:27 AM, Vladimir Ozerov <vozerov@gridgain.com>
> > wrote:
> >
> > > Please see original Sergey's message - when persistence is enabled,
> > memory
> > > is not allocated incrementally, maxSize is used.
> > >
> >
> > Why?
> >
> >
> > > Default settings must allow for normal work on developer's environment.
> > >
> >
> > Agree, but why not in increments?
> >
> >
> > >
> > > ср, 2 авг. 2017 г. в 1:10, Denis Magda <dmagda@apache.org>:
> > >
> > > > > Why not allocate in increments automatically?
> > > >
> > > > This is exactly how the allocation works right now. The memory will
> > grow
> > > > incrementally until the max size is reached (80% of RAM by default).
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Aug 1, 2017, at 3:03 PM, dsetrakyan@apache.org wrote:
> > > > >
> > > > > Vova, 1GB seems a bit too small for me, and frankly i do not want
> t o
> > > > guess. Why not allocate in increments automatically?
> > > > >
> > > > > ⁣D.​
> > > > >
> > > > > On Aug 1, 2017, 11:03 PM, at 11:03 PM, Vladimir Ozerov <
> > > > vozerov@gridgain.com> wrote:
> > > > >> Denis,
> > > > >> No doubts you haven't heard about it - AI 2.1 with persistence,
> when
> > > > >> 80% of
> > > > >> RAM is allocated right away, was released several days ago. How
do
> > you
> > > > >> think, how many users tried it already?
> > > > >>
> > > > >> Guys,
> > > > >> Do you really think allocating 80% of available RAM is a normal
> > thing?
> > > > >> Take
> > > > >> your laptop and check how many available RAM you have right now.
> Do
> > > you
> > > > >> fit
> > > > >> to remaining 20%? If not, then running AI with persistence with
> all
> > > > >> defaults will bring your machine down. This is insane. We shold
> > > > >> allocate no
> > > > >> more than 1Gb, so that user can play with it without any problems.
> > > > >>
> > > > >> On Tue, Aug 1, 2017 at 10:26 PM, Denis Magda <dmagda@apache.org>
> > > wrote:
> > > > >>
> > > > >>> My vote goes for option #1 too. I don’t think that 80%
is too
> > > > >> aggressive
> > > > >>> to bring it down.
> > > > >>>
> > > > >>> IGNITE-5717 was created to fix the issue of the 80% RAM
> allocation
> > on
> > > > >> 64
> > > > >>> bit systems when Ignite works on top of 32 bit JVM. I’ve
not
> heard
> > of
> > > > >> any
> > > > >>> other complaints in regards the default allocation size.
> > > > >>>
> > > > >>> —
> > > > >>> Denis
> > > > >>>
> > > > >>>> On Aug 1, 2017, at 10:58 AM, dsetrakyan@apache.org wrote:
> > > > >>>>
> > > > >>>> I prefer option #1.
> > > > >>>>
> > > > >>>> ⁣D.​
> > > > >>>>
> > > > >>>> On Aug 1, 2017, 11:20 AM, at 11:20 AM, Sergey Chugunov
<
> > > > >>> sergey.chugunov@gmail.com> wrote:
> > > > >>>>> Folks,
> > > > >>>>>
> > > > >>>>> I would like to get back to the question about MemoryPolicy
> > > > >> maxMemory
> > > > >>>>> defaults.
> > > > >>>>>
> > > > >>>>> Although MemoryPolicy may be configured with initial
and
> > maxMemory
> > > > >>>>> settings, when persistence is used MemoryPolicy always
> allocates
> > > > >>>>> maxMemory
> > > > >>>>> size for performance reasons.
> > > > >>>>>
> > > > >>>>> As default size of maxMemory is 80% of physical memory
it
> causes
> > > > >> OOME
> > > > >>>>> exceptions of 32 bit platforms (either on OS or JVM
level) and
> > > > >> hurts
> > > > >>>>> performance in setups when multiple Ignite nodes
are started on
> > > > >> the
> > > > >>>>> same
> > > > >>>>> physical server.
> > > > >>>>>
> > > > >>>>> I suggest to rethink these defaults and switch to
other
> options:
> > > > >>>>>
> > > > >>>>> - Check whether platform is 32 or 64 bits and adapt
defaults.
> In
> > > > >> this
> > > > >>>>> case we still need to address the issue with multiple
nodes on
> > one
> > > > >>>>> machine
> > > > >>>>> even on 64 bit systems.
> > > > >>>>>
> > > > >>>>> - Lower defaults for maxMemory and allocate, for
instance,
> > > > >> max(0.3 *
> > > > >>>>> availableMemory, 1Gb).
> > > > >>>>> This option allows us to solve all issues with starting
on 32
> bit
> > > > >>>>> platforms and reduce instability with multiple nodes
on the
> same
> > > > >>>>> machine.
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Thoughts and/or other options?
> > > > >>>>>
> > > > >>>>> Thanks,
> > > > >>>>> Sergey.
> > > > >>>
> > > > >>>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message