singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa-doc] nudles commented on a change in pull request #7: add onnx doc
Date Sun, 05 Apr 2020 04:01:34 GMT
nudles commented on a change in pull request #7: add onnx doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403645611
 
 

 ##########
 File path: docs-site/docs/onnx.md
 ##########
 @@ -391,178 +535,227 @@ autograd.training = False
 test(new_model, valid_x, valid_y, dev=dev)
 ```
 
-## Example: ONNX tiny_yolov2 on singa
+## ONNX model zoo
 
-Now, the onnx of Singa supports importing models from
-[Onnx Model Zoo](https://github.com/onnx/models). We will show you how to
-inmport a Tiny-Yolo-V2 model and verify the correctness of the model by using
-its test dataset.
+The [ONNX Model Zoo](https://github.com/onnx/models) is a collection of
+pre-trained, state-of-the-art models in the ONNX format contributed by community
+members. SINGA has supported several CV and NLP models now. More models are
+going to be supported soon.
 
-### Load model
+### Image Classification
 
-Firstly, we try to download the Tiny-Yolo-V2 model from the Onnx Model Zoo if it
-doesn't exist already, and then load this model:
-
-```python
-def load_model():
-    url = 'https://onnxzoo.blob.core.windows.net/models/opset_8/tiny_yolov2/tiny_yolov2.tar.gz'
-    download_dir = '/tmp/'
-    filename = os.path.join(download_dir, 'tiny_yolov2', '.', 'Model.onnx')
-    with tarfile.open(check_exist_or_download(url), 'r') as t:
-        t.extractall(path=download_dir)
-    return filename
-
-def check_exist_or_download(url):
-    download_dir = '/tmp/'
-    name = url.rsplit('/', 1)[-1]
-    filename = os.path.join(download_dir, name)
-    if not os.path.isfile(filename):
-        print("Downloading %s" % url)
-        urllib.request.urlretrieve(url, filename)
-    return filename
+This collection of models take images as input, then classifies the major
+objects in the images into 1000 object categories such as keyboard, mouse,
+pencil, and many animals.
 
-dev = device.create_cuda_gpu()
-model_path = load_model()
-onnx_model = onnx.load(model_path)
-```
+| Model Class                                                                           
        | Reference                                          | Description               
                                                                                         
                                                                    | Link               
                                                                                         
                                          |
+| ----------------------------------------------------------------------------------------------
| -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| -------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| <b>[MobileNet](https://github.com/onnx/models/tree/master/vision/classification/mobilenet)</b>
| [Sandler et al.](https://arxiv.org/abs/1801.04381) | Light-weight deep neural network best
suited for mobile and embedded vision applications. <br>Top-5 error from paper - ~10%
                                                             | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1HsixqJMIpKyEPhkbB8jy7NwNEFEAUWAf)
|
+| <b>[ResNet18](https://github.com/onnx/models/tree/master/vision/classification/resnet)</b>
    | [He et al.](https://arxiv.org/abs/1512.03385)      | A CNN model (up to 152 layers).
Uses shortcut connections to achieve higher accuracy when classifying images. <br> Top-5
error from paper - ~3.6%                                        | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1u1RYefSsVbiP4I-5wiBKHjsT9L0FxLm9)
|
+| <b>[VGG16](https://github.com/onnx/models/tree/master/vision/classification/vgg)</b>
          | [Simonyan et al.](https://arxiv.org/abs/1409.1556) | Deep CNN model(up to 19 layers).
Similar to AlexNet but uses multiple smaller kernel-sized filters that provides more accuracy
when classifying images. <br>Top-5 error from paper - ~8% | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/14kxgRKtbjPCKKsDJVNi3AvTev81Gp_Ds)
|
 
-### Set batchsize and prepare model
+### Object Detection
 
-Then since lots of example models don't indicate its batch size, we need to
-update it. After that, we can parse the onnx model into singa model:
+Object detection models detect the presence of multiple objects in an image and
+segment out areas of the image where the objects are detected.
 
-```python
-def update_batch_size(onnx_model, batch_size):
-    model_input = onnx_model.graph.input[0]
-    model_input.type.tensor_type.shape.dim[0].dim_value = batch_size
-    return onnx_model
+| Model Class                                                                           
                           | Reference                                             | Description
                                                                                         
                             | Link                                                      
                                                                                         
   |
+| -----------------------------------------------------------------------------------------------------------------
| ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------
| -------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| <b>[Tiny YOLOv2](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/tiny_yolov2)</b>
| [Redmon et al.](https://arxiv.org/pdf/1612.08242.pdf) | A real-time CNN for object detection
that detects 20 different classes. A smaller version of the more complex full YOLOv2 network.
| [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11V4I6cRjIJNUv5ZGsEGwqHuoQEie6b1T)
|
 
-# set batch size
-onnx_model = update_batch_size(onnx_model, 1)
-sg_ir = sonnx.prepare(onnx_model, device=dev)
-```
+### Face Analysis
 
-### Define inference
+Face detection models identify and/or recognize human faces and emotions in
+given images.
 
-For clearness, we define a Infer functin to hold the model's forward process:
+| Model Class                                                                           
                   | Reference                                          | Description    
                                                                                         
                          | Link                                                         
                                                                                         
|
+| ---------------------------------------------------------------------------------------------------------
| -------------------------------------------------- | -----------------------------------------------------------------------------------------------------------------------------------
| -------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| <b>[ArcFace](https://github.com/onnx/models/tree/master/vision/body_analysis/arcface)</b>
                | [Deng et al.](https://arxiv.org/abs/1801.07698)    | A CNN based model for
face recognition which learns discriminative features of faces and produces embeddings for
input face images. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1qanaqUKGIDtifdzEzJOHjEj4kYzA9uJC)
|
+| <b>[Emotion FerPlus](https://github.com/onnx/models/tree/master/vision/body_analysis/emotion_ferplus)</b>
| [Barsoum et al.](https://arxiv.org/abs/1608.01041) | Deep CNN for emotion recognition trained
on images of faces.                                                                      
 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1XHtBQGRhe58PDi4LGYJzYueWBeWbO23r)
|
 
-```python
-class Infer:
-    def __init__(self, sg_ir):
-        self.sg_ir = sg_ir
-        for idx, tens in sg_ir.tensor_map.items():
-            # allow the tensors to be updated
-            tens.requires_grad = True
-            tens.stores_grad = True
-            sg_ir.tensor_map[idx] = tens
+### Machine Comprehension
 
-    def forward(self, x):
-        return sg_ir.run([x])[0]
+This subset of natural language processing models that answer questions about a
+given context paragraph.
 
-# inference
-autograd.training = False
-model = Infer(sg_ir)
-```
+| Model Class                                                                           
               | Reference                                             | Description     
                                                               | Link                    
                                                                                         
                                     |
+| -----------------------------------------------------------------------------------------------------
| ----------------------------------------------------- | -------------------------------------------------------------------------------
| -------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| <b>[BERT-Squad](https://github.com/onnx/models/tree/master/text/machine_comprehension/bert-squad)</b>
| [Devlin et al.](https://arxiv.org/pdf/1810.04805.pdf) | This model answers questions based
on the context of the given input paragraph. | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1kud-lUPjS_u-TkDAzihBTw0Vqr0FjCE-)
|
 
-### Load dataset, run and verify
+## Supported operators
 
-Finally, we load the test dataset which is provided by Onnx Model Zoo, do the
-inference and verify its correctness.
+The following operators are supported:
 
-```python
-def load_dataset(test_data_dir):
-    # Load inputs
-    inputs = []
-    inputs_num = len(glob.glob(os.path.join(test_data_dir, 'input_*.pb')))
-    for i in range(inputs_num):
-        input_file = os.path.join(test_data_dir, 'input_{}.pb'.format(i))
-        tensor = onnx.TensorProto()
-        with open(input_file, 'rb') as f:
-            tensor.ParseFromString(f.read())
-        inputs.append(numpy_helper.to_array(tensor))
-
-    # Load reference outputs
-    ref_outputs = []
-    ref_outputs_num = len(glob.glob(os.path.join(test_data_dir, 'output_*.pb')))
-    for i in range(ref_outputs_num):
-        output_file = os.path.join(test_data_dir, 'output_{}.pb'.format(i))
-        tensor = onnx.TensorProto()
-        with open(output_file, 'rb') as f:
-            tensor.ParseFromString(f.read())
-        ref_outputs.append(numpy_helper.to_array(tensor))
-    return inputs, ref_outputs
-
-inputs, ref_outputs = load_dataset(os.path.join('/tmp', 'tiny_yolov2', 'test_data_set_0'))
-x_batch = tensor.Tensor(device=dev, data=inputs[0])
-outputs = model.forward(x_batch)
-
-# Compare the results with reference outputs.
-for ref_o, o in zip(ref_outputs, outputs):
-    np.testing.assert_almost_equal(ref_o, o)
-```
+- Conv
+- Relu
+- Constant
+- MaxPool
+- AveragePool
+- Softmax
+- Sigmoid
+- Add
+- MatMul
+- BatchNormalization
+- Concat
+- Flatten
+- Add
+- Gemm
+- Reshape
+- Sum
+- Cos
+- Cosh
+- Sin
+- Sinh
+- Tan
+- Tanh
+- Acos
+- Acosh
+- Asin
+- Asinh
+- Atan
+- Atanh
+- Selu
+- Elu
+- Equal
+- Less
+- Sign
+- Div
+- Sub
+- Sqrt
+- Log
+- Greater
+- HardSigmoid
+- Identity
+- Softplus
+- Softsign
+- Mean
+- Pow
+- Clip
+- PRelu
+- Mul
+- Transpose
+- Max
+- Min
+- Shape
+- And
+- Or
+- Xor
+- Not
+- Neg
+- Reciprocal
+- LeakyRelu
+- GlobalAveragePool
+- ConstantOfShape
+- Dropout
+- ReduceSum
+- ReduceMean
+- LeakyRelu
+- GlobalAveragePool
+- Squeeze
+- Unsqueeze
+- Slice
+- Ceil
+- Split
+- Gather
+- Tile
+- NonZero
+- Cast
+- OneHot
+
+### Special comments for ONNX backend
+
+- Conv, MaxPool and AveragePool
+
+  Input must be 1d`(N*C*H)` and 2d(`N*C*H*W`) shape and `dilation` must be 1.
+
+- BatchNormalization
+
+  `epsilon` is 1e-05 and cannot be changed.
+
+- Cast
+
+  Only support float32 and int32, other types are casted to these two types.
+
+- Squeeze and Unsqueeze
+
+  If you encounter errors when you `Squeeze` or `Unsqueeze` between `Tensor` and
+  Scalar, please report to us.
+
+- Empty tensor Empty tensor is illegal in SINGA.
+
+## Implementation
+
+The code of SINGA ONNX locates at `python/singa/soonx.py`. There are three main
+class, `SingaFrontend` and `SingaBackend` and `SingaRep`. `SingaFrontend`
+translates a SINGA model to ONNX model; `SingaBackend` translates a ONNX model
+to `SingaRep` object which stores all SINGA operators and tensors(the tensor in
+this doc means SINGA `Tensor`); `SingaRep` can be run like a SINGA model.
+
+### SingaFrontend
+
+The entry function of `SingaFrontend` is `singa_to_onnx_model` which also is
+called `to_onnx`. `singa_to_onnx_model` creates the ONNX model, and it also
+create a ONNX graph by using `singa_to_onnx_graph`.
+
+`singa_to_onnx_graph` accepts the output of the model, and recursively iterate
+the SINGA model's graph from the output to get all operators to form a queue.
+The input and intermediate tensors, i.e, trainable weights, of the SINGA model
+is picked up at the same time. The input is stored in `onnx_model.graph.input`;
+the output is stored in `onnx_model.graph.output`; and the trainable weights are
+stored in `onnx_model.graph.initializer`.
+
+Then the SINGA operator in the queue is translated to ONNX operators one by one.
+`_rename_operators` defines the operators name mapping between SINGA and ONNX.
+`_special_operators` defines which function to be used to translate the
+operator.
+
+In addition, some operators in SINGA has different definition with ONNX, that
+is, ONNX regards some attributes of SINGA operators as input, so
+`_unhandled_operators` defines which function to handle the special operator.
+
+Since the bool type is regarded as int32 in SINGA, `_bool_operators` defines the
+operators to be changed as bool type.
+
+### SingaBackend
+
+The entry function of `SingaBackend` is `prepare` which checks the version of
+ONNX model and call `_onnx_model_to_singa_net` then.
+
+The purpose of `_onnx_model_to_singa_net` is to get SINGA tensors and operators.
+The tensors are stored in a dictionary by their name in ONNX, and operators are
+stored in queue by the form of
+`namedtuple('SingaOps', ['name', 'op', 'handle', 'forward'])`. For each
+operator, `name` is its ONNX node name; `op` is the ONNX node; `forward` is the
+SINGA operator's forward function; `handle` is prepared for some special
+operators such as Conv and Pooling which has `handle` object.
+
+The first step of `_onnx_model_to_singa_net` is to call `_init_graph_parameter`
+to get all tensors within the model. For trainable weights, it can init SINGA
+`Tensor` from `onnx_model.graph.initializer`. Please note, the weights may also
+be stored within graph's input or a ONNX node called `Constant`, SINGA can also
+handle these.
+
+Though all weights are stored within ONNX model, the input of the model is
+unknown but its shape and type. So SINGA support two ways to init input, 1,
+generate random tensor by its shape and type, 2, allow the user to assign the
+input. The first way works fine for most models, however, for some model such as
+bert, the indices of matrix cannot be random generated otherwise it will incurs
+errors.
+
+Then, `_onnx_model_to_singa_net` iterators all nodes within ONNX graph to
+translate it to SIGNA operators. Also, `_rename_operators` defines the operators
+name mapping between SINGA and ONNX. `_special_operators` defines which function
+to be used to translate the operator. `_run_node` runs the generated SINGA model
+by its input tensors and store its output tensors for being used by later
+operators.
 
-## Supported operators
+This class finally return a `SingaRep` object and stores all SINGA tensors and
+operators within it.
 
-The following operators are supported:
+### SingaRep
 
-| Operation          | Comments                                  |
-| ------------------ | ----------------------------------------- |
 
 Review comment:
   are all onnx operators supported?
   if not, we still need this list.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message