singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Diganta Misra (Jira)" <>
Subject [jira] [Created] (SINGA-498) Add Mish Activation Function
Date Wed, 20 Nov 2019 10:27:00 GMT
Diganta Misra created SINGA-498:

             Summary: Add Mish Activation Function
                 Key: SINGA-498
             Project: Singa
          Issue Type: New Feature
          Components: Core
            Reporter: Diganta Misra
         Attachments: se50_1.png

Mish is a new novel activation function proposed in this [paper](
It has shown promising results so far and has been adopted in several packages including:

- [TensorFlow-Addons](
- [SpaCy (Tok2Vec Layer)](
- [Thinc - SpaCy's official NLP based ML library](
- [Echo AI](
- [Eclipse's deeplearning4j](
- [CNTKX - Extension of Microsoft's CNTK](
- [FastAI-Dev](
- [Darknet](
- [Yolov3](
- [BeeDNN - Library in C++](
- [Gen-EfficientNet-PyTorch](
- [dnet](

All benchmarks, analysis and links to official package implementations can be found in this

It would be nice to have Mish as an option within the activation function group.

The attached image is the comparison of Mish with other conventional activation functions
in a SEResNet-50 for CIFAR-10. 

This message was sent by Atlassian Jira

View raw message