singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <>
Subject Re: Question about model parallelism and multi-GPUs
Date Wed, 11 Nov 2015 00:44:20 GMT
NYCTMI, I am having all doubts.. RNN model parallel described in
tensorFlow whitepaper somewhat makes sense to me but did google
actually uses Model parallel for CNN on multi nodes (equipped
multi-devices) cluster. Blocking mat-mult on GPU appears to me slow
and memory demanding. I mean it's possible but performance will

So, I wanted to check how you approach this problem.

On Wed, Nov 11, 2015 at 9:33 AM, ooibc <> wrote:
> fyi, in case you are not in the dev@ list.
> There is a typo below: ancestor-->descendant (MXNet was introduced on 9/28).
> -------- Original Message --------
> Subject: development plan for SINGA
> Date: 2015-11-10 22:28
> From: ooibc <>
> To:
> Reply-To:
> Based on our quick check at the release of Tensorflow and online
> discussions, it appears to be 2x slower than MXNet (ancestor of CXXNET) on
> cifar10 dataset, and it contains older
> codes like cudnn-v2. This could be just a form of crowdsourcing at work.
> SINGA is data flow centric in design, and provides simple interfaces, from
> layer abstraction to neural net structure, model configuration/mapping,
> model/data partitioning, function overriding, and training framework
> configuration.
> So, we are good and we should keep to the development/release plan outlined
> in
> Thanks, and Happy Deepavali (to those who celebrate)!
> regards
> beng chin

Best Regards, Edward J. Yoon

View raw message