singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lee Chonho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SINGA-81) Helper API, which enables users to construct JobProto in C++ and by Python binding
Date Sun, 04 Oct 2015 04:49:27 GMT

    [ https://issues.apache.org/jira/browse/SINGA-81?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942553#comment-14942553
] 

Lee Chonho commented on SINGA-81:
---------------------------------

With this API, we try to provide a easier and more intuitive way to construct model (i.e.,
job.conf). I mean the API take care of calling add_layer(), set_param(), etc. for users. Later,
users may only use python binding in an interactive manner to run singa.

Still discussing... but
Here is an example of construct all hidden layers between Parser and Softmax layer
by 1 line although there are many optional arguments. Using Boost library, it should be possible.
{code}
N4 = AddLayer(kInnerProduct, _src="parser",
        _num_stack=3, _num_output={2000,1000,500},
	_pname={“w”,"b"}, _init_ptype=kUniform, _low=-0.05, _high=0.05,
	_activation=kSTanh);
{code}

This line will set a part of NetProto like
{code}
  layer{
    name: "fc"
    type: kInnerProduct
    srclayers:"mnist"
    innerproduct_conf{
      num_output: 2000
    }
    param{
      name: "w"
      init {
        type: kUniform
        low:-0.05
        high:0.05
      }
    }
    param{
      name: "b"
      init {
        type : kUniform
        low: -0.05
        high:0.05
      }
    }
  }
  layer{
    name: "tanh"
    type: kSTanh
    srclayers:"fc"
  }
 ....  // next set of hidden layers similar to above
 ....  // another set of hidden layers similar to above
{code}

> Helper API, which enables users to construct JobProto in C++ and by Python binding
> ----------------------------------------------------------------------------------
>
>                 Key: SINGA-81
>                 URL: https://issues.apache.org/jira/browse/SINGA-81
>             Project: Singa
>          Issue Type: New Feature
>            Reporter: Lee Chonho
>
> Proposed design v1
> - (1) have a class named Builder
> - (2) use Boost::parameter library (+ associated necessary header files)
> (1)
> Builder class implements api-like functions to configure JobProto including NetProto
(LayerProto), ClusterProto, UpdaterProto, etc.
> Two options
> a. users call Builder's functions in main.cc like
> {code}
> JobProto jobproto;
> Builder builder( &jobproto, "job name" );
> builder.xxx(xxx)  // add data layer
> builder.xxx(xxx)  // add parser layer
> ... etc. ...
> {code}
> b. we set main.cc like below and users call Builders functions in Construct() 
> {code}
> JobProto jobproto;
> Builder builder( &jobproto, "model name" );
> builder.Construct()
> {code}
> (2)
> Planning to use header-only files from Boost library
> - if the necessary files are small enough
> - because we can use "named arguments" feature with no restriction of # of arguments,
order, types.
> - because function will be intuitive, and adding users' own proto in a straightforward
way.
> Example is here. http://theboostcpplibraries.com/boost.parameter
> By following the example, we can do like
> {code}
> BOOST_PARAMETER_MEMBER_FUNCTION(   
>  (char*), AddLayerData,   tag,   
>  (required     
>    (type, (int))  (name, (char*))  (src, (char*))
>  ) 
>  (optional     
>    (path, (char*), *) (bsize (int) *)
>  ) )
>  {
>     ...  // set values
>     return name;
>  }
> {code}
> Then, users can add a datalayer by
> {code}
> L1 = builder.AddLayerData(kShardData, "data", null, _path="train_shard", _bsize=1000);
> {code}
> (TODO)
> - make use of google protobuf reflection for efficient parameter setting
> - need to avoid multiple calls for adding same/similar layers
> Any suggestion, design idea, comments please.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message