singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SINGA-36) Clean ModelProto, ClusterProto and JobProto
Date Thu, 23 Jul 2015 06:50:04 GMT

    [ https://issues.apache.org/jira/browse/SINGA-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638313#comment-14638313
] 

ASF subversion and git services commented on SINGA-36:
------------------------------------------------------

Commit 0478e8cfdb54688c7c627cd328af04aa6aa30a63 in incubator-singa's branch refs/heads/master
from Wei Wang
[ https://git-wip-us.apache.org/repos/asf?p=incubator-singa.git;h=0478e8c ]

SINGA-36 Clean ModelProto, ClusterProto, JobProto and driver program

The ClusterProto and ModelProto are merged into JobProto.
A SingaProto is added for global singa configuration including zookeeper info, log dir setting.

The driver program (i.e., main.cc) cleaned.
1. Users include singa.h and their own headers the the drvier program.
2. They prepare the job configuration
and singa configuration (may make this transparent to users, e.g., through environment variable,
SINGA_HOME).
3. They register the layers them implemented.
4. Call the trainer to start the training
After linking the driver program with libsinga, users pass the executable to bin/singa-run.sh
together with the workspace where job.conf is stored to submit the job

Some layers are moved into optional_layer.cc(.h) if they have dependencies on external libs
(e.g., lmdb and opencv).
If users do not need these layers, they do not need to install the dependencies. TODO update
configure script to add
these options. USE_OPTIONAL_LAYER controls the compliation of optional_layer.cc(.h).

Move command line argument singa_conf into trainer.cc.
Usually, users do not need to pass this argument explicitly.
The sing-run.sh script will find it automatically.

modify scripts to support new singa command line

minor change on example job config

add SubmitJob func in singa.h to simplify the job submission API.


> Clean ModelProto, ClusterProto and JobProto
> -------------------------------------------
>
>                 Key: SINGA-36
>                 URL: https://issues.apache.org/jira/browse/SINGA-36
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
>
> Currently, we use the google protocol buffer to generate ClusterProto and ModelProto
classes for cluster (i.e., worker, server, group, etc) configuration and model (i.e., neuralnet,
updater, etc) configuration respectively. Theses classes provide functions to load/parse plain
text configuration files.
> To make the naming more representative and simplify the configuration process, this ticket
will:
> * move worker, server and group configuration into JobProto, i.e., these configurations
are job/application specific.
> * move ModelProto as a field of JobProto, because the neuralnet configuration is also
job specific.
> * move zookeeper, hostfile, etc configuration into ClusterProto, because these fields
are shared by all jobs.
> The configuration for ClusterProto is done when installing SINGA and is done for only
once. The configuration file is fixed at $SINGA/conf/singa.conf.
> The configuration for JobProto is done every time a new job is submitted. The conf file
is $workspace/job.conf. Users submit the job by passing -conf=$workspace to ./bin/singa-run.sh



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message