mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@googlemail.com.INVALID>
Subject Re: Off-Heap Memory Management in MXNet Scala
Date Wed, 12 Sep 2018 08:11:59 GMT
Interesting and detailed document!

The JVM garbage collector gets executed depending on the memory pressure
for CPU RAM (or a different custom strategy). It was mentioned that this
document also supports disposing GPU objects. Sorry if I missed it, but how
exactly are we ensuring we don't run out of memory on GPU?

There are quite a lot of cases where we get close to the limit of a
available GPU RAM even with explicit disposes. If we run out of memory, we
get a fatal exception and it's basically game over (as of the current
state). How do handle these cases where we can't rely on paging and the
benefits of virtual memory?

Best regards,
Marco

Timur Shenkao <tsh@timshenkao.su> schrieb am Mi., 12. Sep. 2018, 09:59:

> Thanks for great job!
>
> My questions / proposals.
> 1) Have considered Java collections with low memory footprint like
> Fastutil, Koloboke, etc.? They are much more memory efficient and they have
> "better correspondence" with low level data types.
> 2) MNIST example on the page is "bad" because MNIST is handled pretty fast
> even on laptop, i.e. we won't catch GC & off-heap problems.
> 3) Is it 1.2.0-java branch where Java API things happen?
> 4) System.gc() behaves differently on various JVM platforms, JDK
> implementations, GC types. So, I am sure that we will get users' requests
> to eliminate this approach in the future.
> 5) Frameworks like mxnet aren't used separately. Folks have to integrate
> Spark or Spring with DL libraries. And in this case, they often use CMS or
> even more archaic GCs as for streaming or long living jobs G1GC isn't
> always good.
>
>
>
> On Wednesday, September 12, 2018, Chris Olivier <cjolivier01@gmail.com>
> wrote:
>
> > do you log on finalize() if the object wasn’t properly freed (ie
> > NDArray.finalize())? is that available in Scala?
> >
> > On Tue, Sep 11, 2018 at 6:12 PM Qing Lan <lanking520@live.com> wrote:
> >
> > > Nice document! Way better than current .dispose() in Scala!
> > >
> > > Thanks,
> > > Qing
> > >
> > > On 9/11/18, 6:04 PM, "Chris Olivier" <cjolivier01@gmail.com> wrote:
> > >
> > >     wow, incredible document!
> > >
> > >     On Tue, Sep 11, 2018 at 2:37 PM Naveen Swamy <mnnaveen@gmail.com>
> > > wrote:
> > >
> > >     > Hi All,
> > >     >
> > >     > I am working on managing Off-Heap Memory Management and have
> > written
> > > a
> > >     > proposal here based on my prototype and research I did.
> > >     >
> > >     > Please review the doc and provide your feedback ?
> > >     >
> > >     >
> > >
> https://cwiki.apache.org/confluence/display/MXNET/JVM+Memory+Management
> > >     >
> > >     > I had offline discussion with a few people I work with and added
> > > their
> > >     > feedback to the doc as well.
> > >     >
> > >     > Thanks, Naveen
> > >     >
> > >
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message