singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Question about model parallelism and multi-GPUs
Date Wed, 11 Nov 2015 00:27:56 GMT
Really awesome :o thanks!

On Tue, Nov 10, 2015 at 8:30 PM, ooibc <ooibc@comp.nus.edu.sg> wrote:
> On 2015-11-10 15:55, Edward J. Yoon wrote:
>>
>> Hi forks,
>>
>> I have a question about model parallelism and multi-GPUs. 1) Singa is
>> going to support both model parallelism and multi-GPUs?
>
>
> Yes, based on schedule outlined in
> https://singa.incubator.apache.org/develop/schedule.html
>
>
>> 2) If so,
>> what's the model partition strategy and GPU acceleration strategy?
>
>
> The general strategy is described in the ppt:
> www.comp.nus.edu.sg/~ooibc/mm2015.ppt
>
> For CPU-GPU computation, we will optimize the communication and memory
> management, and
> also, introduce a new SGD computation (this is being tested as an academic
> paper at this moment).
>
> We will focus more on CPU optimization so that distributed DL training can
> be run on
> existing clusters efficiently.
>
>>
>> Thanks!
>
>
>



-- 
Best Regards, Edward J. Yoon

Mime
View raw message