singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ngin Yun Chuan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SINGA-413) Hyper-parameter configuration API
Date Sun, 02 Dec 2018 09:27:00 GMT

    [ https://issues.apache.org/jira/browse/SINGA-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706150#comment-16706150
] 

Ngin Yun Chuan edited comment on SINGA-413 at 12/2/18 9:26 AM:
---------------------------------------------------------------

Hi [~wangwei.cs], I've pushed the API changes to the [`improve_knob_api` branch|https://github.com/nginyc/rafiki/compare/improve_knob_api].
Do you have any feedback on the revised model knob configuration API before I finish it up?
The worker-side implementation for this improvement is not complete yet - wanted to get your
feedback first.



was (Author: nginyc):
Hi [~wangwei.cs], I've pushed the API changes to the [`improve_knob_api` branch|https://github.com/nginyc/rafiki/compare/improve_knob_api].
Do you have any feedback on the revised model knob configuration API before continuing? The
worker-side implementation for this improvement is not complete yet - wanted to get your feedback
first.


> Hyper-parameter configuration API
> ---------------------------------
>
>                 Key: SINGA-413
>                 URL: https://issues.apache.org/jira/browse/SINGA-413
>             Project: Singa
>          Issue Type: Improvement
>            Reporter: wangwei
>            Priority: Major
>
> The current API for hyper-parameter configuration in Rafiki requires the model contributor
to implement
> {code:java}
> def get_knob_config(self):
>     {
>     'knobs': {
>            'hidden_layer_units': {
>            'type': 'int',
>            'range': [2, 128]
>            },
>      ...
>      }
> def init(self, knobs):
>     self.hidden_layer_units = knobs.get(hidden_layer_units){code}
> The json style can be replaced by
> {code}
> def check(knobs, val):
>    if knobs[0].value() == 4 and val == 4:
>       return False
>    else:
>       return True
> def register_knobs(self):
>    self.hidden_layer_units = Knob(32, kInt, range=(2, 128))
>    self.learning_rate = Knob(0.02, kExpFloat, range=(1e-1, 1e-4))
>    self.batch_size = Knob(8, kIntCat, range = [4, 8, 16]))
>    self.num_epoch = Knob(4, kIntCat, range = [4, 8, 16], depends=self.batch_size, callback=check()))
> def train(self):
>    for i in range(self.num_epoch.value()):
>       ...{code}
> The worker's workflow is like
> {code:java}
> for trial in range(total_trials):
>    model = Model()
>    model.register_knobs()
>    knobs = [x for x, y in model.__dict__.getitems() if isinstance(y, Knob)]
>    if trial == 0:
>      reigster_knobs_to_advisor(knobs)
>    get_knobs_from_advisor(knobs){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message