singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yin Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SINGA-407) Singa example used up all memory and hangs
Date Fri, 23 Nov 2018 06:57:00 GMT

    [ https://issues.apache.org/jira/browse/SINGA-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696439#comment-16696439
] 

Yin Xu commented on SINGA-407:
------------------------------

Hi, I have updated the version and it still used up all the memory. But I see the process
is killed and the training job does not complete when I returned to check: 

conv3_1_3-->drop3_1: 0.064721
drop3_1-->conv3_2_1: 0.513878
conv3_2_1-->conv3_2_2: 0.223191
conv3_2_2-->conv3_2_3: 0.082423
Killed
xuyin@xuyin-nusszai:~/workspace/incubator-singa/examples/cifar10$

> Singa example used up all memory and hangs
> ------------------------------------------
>
>                 Key: SINGA-407
>                 URL: https://issues.apache.org/jira/browse/SINGA-407
>             Project: Singa
>          Issue Type: Bug
>          Components: Application
>            Reporter: Yin Xu
>            Priority: Major
>         Attachments: 20181119_055525455_iOS.jpg
>
>
> I installed singa on my machine and run the exapmles
> [https://github.com/apache/incubator-singa/tree/master/examples/cifar10]
> I simply run {{python train.py vgg cifar-10-batches-py.}}
> {{It runs fine initially, but it keep using the memory and finally used up all the memory
and swap, then the machine hangs.}}
> {{My machine is Ubuntu 18.04, with kernel 4.15.0-39-generic}}
> {{GPU card is: GeForce GTX 1060}}
> {{The singa version is 1.2.0 py36_cuda9.0_cudnn7.1.2}}
> {{Attach I show the GPU and resource usage when it hangs }}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message