hivemall-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Makoto Yui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVEMALL-222) Introduce Gradient Clipping to avoid exploding gradient to General Classifier/Regressor
Date Wed, 24 Oct 2018 06:33:00 GMT

    [ https://issues.apache.org/jira/browse/HIVEMALL-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661796#comment-16661796
] 

Makoto Yui commented on HIVEMALL-222:
-------------------------------------

Clip by value
[https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/clip_ops.py#L37
] [https://github.com/scikit-learn/scikit-learn/blob/0fc7ce6bb8bb6b5b98c66ad2c0a009753def945a/sklearn/linear_model/sgd_fast.pyx#L701]


Clip by l2 norm
[https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/clip_ops.py#L110
] [http://www.cs.toronto.edu/~rgrosse/courses/csc321_2017/readings/L15%20Exploding%20and%20Vanishing%20Gradients.pdf]

 

 

> Introduce Gradient Clipping to avoid exploding gradient to General Classifier/Regressor
> ---------------------------------------------------------------------------------------
>
>                 Key: HIVEMALL-222
>                 URL: https://issues.apache.org/jira/browse/HIVEMALL-222
>             Project: Hivemall
>          Issue Type: Improvement
>    Affects Versions: 0.5.0
>            Reporter: Makoto Yui
>            Assignee: Makoto Yui
>            Priority: Minor
>             Fix For: 0.5.2
>
>
> Gradient Clipping is useful for avoiding exploding gradients
> [https://github.com/scikit-learn/scikit-learn/blob/0fc7ce6bb8bb6b5b98c66ad2c0a009753def945a/sklearn/linear_model/sgd_fast.pyx#L701]
> So, implement it for General Classifier/Regressor



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message