apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Farkas <...@datatorrent.com>
Subject Re: Default App Master Container Memory Size
Date Thu, 24 Dec 2015 01:04:23 GMT
If there are no objections I will open a pull request shortly to bump up
the default App Master Memory to 1280 MB. I recently got burned by this
default AGAIN when the App Master randomly died in the middle of demo. The
demo application was small and only had 6 physical operators.

Thanks,
Tim

On Mon, Dec 14, 2015 at 2:01 PM, Munagala Ramanath <ram@datatorrent.com>
wrote:

> Sorry, meant to say 4-6GB, not MB.
>
> Ram
>
> On Mon, Dec 14, 2015 at 2:00 PM, Munagala Ramanath <ram@datatorrent.com>
> wrote:
>
> > I'm OK with increasing it to 1280MB.
> >
> > I agree with Sandeep about new people trying out Apex,
> > often on limited environments like VMs limited to 4-6MB. With a default
> of
> > 0.5GB per Buffer Server
> > and 1GB per operator, things start to fail when the user has more than a
> > handful of operators.
> >
> > I'm guessing it's not just the number of the operators that puts memory
> > pressure on master memory
> > but the size of the jars that get pulled in by all the operator classes
> > and their dependencies. If this is
> > true, one could cause failure even with a single, sufficiently complex
> > operator.
> >
> > Ram
> >
> > On Mon, Dec 14, 2015 at 1:41 PM, Timothy Farkas <tim@datatorrent.com>
> > wrote:
> >
> >> Hi Sandeep,
> >>
> >> I have seen the issue even with small apps like Twitter Demo. It happens
> >> less frequently in very small apps but the issue is still there. It
> >> happens
> >> frequently in medium size apps (10 containers). Could we increase the
> >> MASTER_MEMORY_MB to 1280 instead of 2048? That way we can get greater
> >> stability with the defaults settings without allocating too much
> >> additional
> >> memory?
> >>
> >> Thanks,
> >> Tim
> >>
> >> On Sun, Dec 13, 2015 at 1:22 AM, Sandeep Deshmukh <
> >> sandeep@datatorrent.com>
> >> wrote:
> >>
> >> > With large number of containers, the problem becomes severe and I too
> >> have
> >> > faced it.
> >> >
> >> > Not sure if default should be increased as that should work in most of
> >> the
> >> > cases and even when new people are trying out Apex for the first time.
> >> > They may not have enough memory.
> >> >
> >> > Regards
> >> > Sandeep
> >> > On 13 Dec 2015 11:30, "Ganelin, Ilya" <Ilya.Ganelin@capitalone.com>
> >> wrote:
> >> >
> >> > > Tim - great point, we reduce the memory consumption for operators
> and
> >> do
> >> > > actually increase it for the app master, especially when we have
> more
> >> > > operators. I misread your original email.
> >> > >
> >> > >
> >> > >
> >> > > Thank you,
> >> > > Ilya Ganelin
> >> > >
> >> > >
> >> > >
> >> > > -----Original Message-----
> >> > > From: Timothy Farkas [tim@datatorrent.com<mailto:
> tim@datatorrent.com
> >> >]
> >> > > Sent: Saturday, December 12, 2015 11:30 PM Eastern Standard Time
> >> > > To: dev@apex.incubator.apache.org
> >> > > Subject: Re: Default App Master Container Memory Size
> >> > >
> >> > >
> >> > > Hi Ilya,
> >> > >
> >> > > Dou you reduce the memory to 256 mb for operators only or for the
> App
> >> > > Master as well?
> >> > >
> >> > > Thanks,
> >> > > Tim
> >> > >
> >> > > On Sat, Dec 12, 2015 at 8:06 PM, Ganelin, Ilya <
> >> > > Ilya.Ganelin@capitalone.com>
> >> > > wrote:
> >> > >
> >> > > > Tim - in our applications we've actually tried to go with smaller
> >> > > > containers, as small as 256 MB, simply because many operators
> don't
> >> > > demand
> >> > > > any memory and this allows us to run more partitions.
> >> > > >
> >> > > > We haven't noticed performance degradation.
> >> > > >
> >> > > >
> >> > > >
> >> > > > Thank you,
> >> > > > Ilya Ganelin
> >> > > >
> >> > > >
> >> > > >
> >> > > > -----Original Message-----
> >> > > > From: Timothy Farkas [tim@datatorrent.com<mailto:
> >> tim@datatorrent.com>]
> >> > > > Sent: Saturday, December 12, 2015 11:02 PM Eastern Standard Time
> >> > > > To: dev@apex.incubator.apache.org
> >> > > > Subject: Default App Master Container Memory Size
> >> > > >
> >> > > >
> >> > > > Hi All,
> >> > > >
> >> > > > The default value for MASTER_MEMORY_MB is 1024. I've observed
in
> >> > multiple
> >> > > > demo apps and in the production apps of some of our users that
> 1024
> >> MB
> >> > is
> >> > > > not enough memory, and the app master dies. This was observed
on
> >> small
> >> > > > applications with 5 - 20 containers. I think the default app
> master
> >> > > memory
> >> > > > size should be 2048.
> >> > > >
> >> > > > Any feedback?
> >> > > >
> >> > > > Thanks,
> >> > > > Tim
> >> > > > ________________________________________________________
> >> > > >
> >> > > > The information contained in this e-mail is confidential and/or
> >> > > > proprietary to Capital One and/or its affiliates and may only
be
> >> used
> >> > > > solely in performance of work or services for Capital One. The
> >> > > information
> >> > > > transmitted herewith is intended only for use by the individual
or
> >> > entity
> >> > > > to which it is addressed. If the reader of this message is not
the
> >> > > intended
> >> > > > recipient, you are hereby notified that any review,
> retransmission,
> >> > > > dissemination, distribution, copying or other use of, or taking
of
> >> any
> >> > > > action in reliance upon this information is strictly prohibited.
> If
> >> you
> >> > > > have received this communication in error, please contact the
> sender
> >> > and
> >> > > > delete the material from your computer.
> >> > > >
> >> > > ________________________________________________________
> >> > >
> >> > > The information contained in this e-mail is confidential and/or
> >> > > proprietary to Capital One and/or its affiliates and may only be
> used
> >> > > solely in performance of work or services for Capital One. The
> >> > information
> >> > > transmitted herewith is intended only for use by the individual or
> >> entity
> >> > > to which it is addressed. If the reader of this message is not the
> >> > intended
> >> > > recipient, you are hereby notified that any review, retransmission,
> >> > > dissemination, distribution, copying or other use of, or taking of
> any
> >> > > action in reliance upon this information is strictly prohibited. If
> >> you
> >> > > have received this communication in error, please contact the sender
> >> and
> >> > > delete the material from your computer.
> >> > >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message