horn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Parallelizing SGD for CNN model
Date Tue, 13 Oct 2015 10:56:20 GMT
You're right. I mean, there're some techniques that using conv-deconv
together for semantic segmentation. So, 2 times layers and parameters
are needed. I heard that it was difficult to learn by memory
limitation from them.

On Tue, Oct 13, 2015 at 7:52 PM, Shubham Mehta
<shubham.mehta93@gmail.com> wrote:
> Hi,
>
> As far as I understood De-convolutional NN (DNN) also have same number of
> parameters as Convolutional NN (CNN). The only difference between two is
> Deconvolutional uses filters for deconvoluting image and generated feature
> map is transferred to next layer while Convolutional uses filters to
> convolute with image and resultant is transferred to next layer.
>
> Even in DNN ( CNN ) computation is much more in deconvolution layers (
> convolutional layers ) as compared to fully connected layers and parameters
> are much lesser. So the architecture that we use for DNN will be similar to
> what we use for CNN. Only the operation at each Neuron/Layer changes.
>
> Please correct me if I'm wrong in my understanding.
>
> Regards,
> Shubham
>
>
>
> On Tue, Oct 13, 2015 at 7:22 PM, Edward J. Yoon <edwardyoon@apache.org>
> wrote:
>
>> Hi,
>>
>> There's also a de-convolutional networks that requires huge memory for
>> large number of parameters.
>>
>> On Tue, Oct 13, 2015 at 6:46 PM, Shubham Mehta
>> <shubham.mehta93@gmail.com> wrote:
>> > Hello, everyone
>> >
>> > A very good read on parallelizing Convolutional Neural Network.
>> >
>> > http://arxiv.org/abs/1404.5997
>> >
>> > Apache Singa follows this approach specifically for CNN.
>> >
>> > The gist of paper is that data parallelism is more beneficial for
>> > Convolutional layers while model parallelism for fully-connected layers.
>> >
>> >
>> > Regards
>> > Shubam Mehta
>> >
>> > --
>> > Shubham Mehta
>> > B.Tech 2015
>> > Computer Science and Engineering
>> > IIT Bombay
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>>
>
>
>
> --
> Shubham Mehta
> B.Tech 2015
> Computer Science and Engineering
> IIT Bombay



-- 
Best Regards, Edward J. Yoon

Mime
View raw message