mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timur Shenkao <...@timshenkao.su>
Subject Re: Off-Heap Memory Management in MXNet Scala
Date Wed, 12 Sep 2018 07:59:20 GMT
Thanks for great job!

My questions / proposals.
1) Have considered Java collections with low memory footprint like
Fastutil, Koloboke, etc.? They are much more memory efficient and they have
"better correspondence" with low level data types.
2) MNIST example on the page is "bad" because MNIST is handled pretty fast
even on laptop, i.e. we won't catch GC & off-heap problems.
3) Is it 1.2.0-java branch where Java API things happen?
4) System.gc() behaves differently on various JVM platforms, JDK
implementations, GC types. So, I am sure that we will get users' requests
to eliminate this approach in the future.
5) Frameworks like mxnet aren't used separately. Folks have to integrate
Spark or Spring with DL libraries. And in this case, they often use CMS or
even more archaic GCs as for streaming or long living jobs G1GC isn't
always good.



On Wednesday, September 12, 2018, Chris Olivier <cjolivier01@gmail.com>
wrote:

> do you log on finalize() if the object wasn’t properly freed (ie
> NDArray.finalize())? is that available in Scala?
>
> On Tue, Sep 11, 2018 at 6:12 PM Qing Lan <lanking520@live.com> wrote:
>
> > Nice document! Way better than current .dispose() in Scala!
> >
> > Thanks,
> > Qing
> >
> > On 9/11/18, 6:04 PM, "Chris Olivier" <cjolivier01@gmail.com> wrote:
> >
> >     wow, incredible document!
> >
> >     On Tue, Sep 11, 2018 at 2:37 PM Naveen Swamy <mnnaveen@gmail.com>
> > wrote:
> >
> >     > Hi All,
> >     >
> >     > I am working on managing Off-Heap Memory Management and have
> written
> > a
> >     > proposal here based on my prototype and research I did.
> >     >
> >     > Please review the doc and provide your feedback ?
> >     >
> >     >
> > https://cwiki.apache.org/confluence/display/MXNET/JVM+Memory+Management
> >     >
> >     > I had offline discussion with a few people I work with and added
> > their
> >     > feedback to the doc as well.
> >     >
> >     > Thanks, Naveen
> >     >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message