mxnet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-mxnet] ArmageddonKnight commented on issue #14973: [MXNET-1404] Added the GPU memory profiler
Date Fri, 24 May 2019 05:51:38 GMT
ArmageddonKnight commented on issue #14973: [MXNET-1404] Added the GPU memory profiler
URL: https://github.com/apache/incubator-mxnet/pull/14973#issuecomment-495481289
 
 
   Hi @anirudh2290 ,
   
   Thanks for your valuable feedback. Let me record a list of Discussion and TODO items:
   
   ## Discussion
   
   - [ ] Preprocessor Directives (1. 2.). 
   - [ ]
   - [ ] Imperative Support (4.). We should **NOT** drop the support for imperative, as this
will significantly boost the amount of GPU memory allocated with the unknown tag. For instance,
currently most optimizer states are allocated using the pure imperative approach. In fact,
almost all the current optimizer implementations (e.g., SGD, Adam) initialize the optimizer
states with `mx.nd.zero`. If we drop the imperative support, then all of those allocations
will fall to the `unknown` category, which can be `1 GB` for large models.
   - [ ] Profiler API Integration (5. 6. 7. 8.). The GPU memory profiler is different from
the existing profilers in many ways: (1) It is not using the `chrome://tracing` visualization
backend, and the reason is because it needs to accept the users' input on defining the **keyword
dictionaries for grouping storage tags** (also, I do not see a very good way of visualizing
percentage using `chrome://tracing`). (2) Because it 
   
   ## TODO 
   
   - [ ] Add minimum working example in the `example` directory to show how `SETME`, analyzer,
and plotter work.
   - [ ]

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message