mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YiZhi Liu <javeli...@gmail.com>
Subject Re: mxnet Scala Convolution
Date Thu, 19 Oct 2017 08:20:19 GMT
You can retrieve the functions/operators by reflection. Or, when
compiling the scala package, it prints all operators out.

2017-10-19 8:15 GMT+00:00 YiZhi Liu <javelinjs@gmail.com>:
> The javadoc cannot be added in this way. I'm afraid we have to write
> another 'javadoc generator' for these operators.
>
> 2017-10-19 7:53 GMT+00:00 TongKe Xue <tkxue@tkxue.org>:
>> HI YiZhi,
>>
>>
>> 1. I looked at:
>> scala-package/macros/src/main/scala/ml/dmlc/mxnet/SymbolMacro.scala
>> scala-package/macros/src/main/scala/ml/dmlc/mxnet/NDArrayMacro.scala
>>
>> These files appear to dynamically register nnvm functions into
>> NDArray/Symbol right?
>>
>> I would like the "javadocs" of the dynamically added functions.
>>
>>
>> 2. I looked at mxnet-macros_2.11-0.11.0-SNAPSHOT-javadoc.jar
>>
>> That jar looks mostly empty. Is that expected, or am I doing something wrong?
>>
>>
>> 3. Currently, the best way I know for retrieving all the functions is
>> to fire up a repl and run reflection. Is there a better method?
>>
>>
>> Thanks,
>> --TongKe
>>
>>
>>
>> On Wed, Oct 18, 2017 at 8:59 PM, YiZhi Liu <javelinjs@gmail.com> wrote:
>>> Hi TongKe,
>>>
>>> The symbols you are looking for are auto-generated by scala macros.
>>> Pls refer to scala-package/macros
>>>
>>> 2017-10-19 0:40 GMT+00:00 TongKe Xue <tkxue@tkxue.org>:
>>>> Hi Rahul,
>>>>
>>>>   Thanks for explaining the high level design + pointing to the
>>>> implementation details.
>>>>
>>>>   Besides reading the C++ code and mentally translating the Scala
>>>> calls, is there a way to get a list of all generated Scala functions?
>>>>
>>>>   I have looked at:
>>>>
>>>> 1. https://mxnet.incubator.apache.org/api/scala/symbol.html
>>>> shows a few examples, but is not exhaustive
>>>>
>>>> 2. https://mxnet.incubator.apache.org/api/scala/docs/index.html#ml.dmlc.mxnet.Symbol
>>>> appears more comprehensive, but I find neither Convolution nor Softmax there.
>>>>
>>>>
>>>> More specifically, my question is: nnvm adds a bunch of Scala bindings
>>>> to C++ code. How do I get a list of all these bindings (name, type of
>>>> inputs, type of output).
>>>>
>>>>
>>>> Thanks!
>>>> --TongKe
>>>>
>>>>
>>>> On Wed, Oct 18, 2017 at 5:28 PM, Rahul Huilgol <rahulhuilgol@gmail.com>
wrote:
>>>>> Hi TongKe,
>>>>>
>>>>> These are operators defined in the c++ backend under src/operator. For
>>>>> example convolution is here
>>>>> https://github.com/apache/incubator-mxnet/blob/master/src/operator/convolution.cc
>>>>> . The operators are registered using nnvm, which helps automatically
>>>>> generate the frontend functions.
>>>>>
>>>>> This tutorial on how to add a backend operator
>>>>> <https://github.com/apache/incubator-mxnet/blob/master/docs/how_to/add_op_in_backend.md>
>>>>> contains information on how to register such operators, which would help
>>>>> you understand the above file.
>>>>> An excerpt from there (for quadratic operator) : "If you use python,
when
>>>>> you type import mxnet as mx, two python functions for invoking your backend
>>>>> implementation are generated on the fly: one is for imperative programming
>>>>> registered as mxnet.ndarray.quadratic or mx.nd.quadratic for short; the
>>>>> other one is for symbolic programming registered under module
>>>>> mxnet.symbol.quadratic or mx.sym.quadratic for short."
>>>>>
>>>>> I'd think the Scala package works similarly.
>>>>>
>>>>> Regards,
>>>>> Rahul
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Oct 18, 2017 at 5:06 PM, TongKe Xue <tkxue@tkxue.org> wrote:
>>>>>
>>>>>> My earlier question was a bit messy.
>>>>>>
>>>>>> To rephrase my question:
>>>>>>
>>>>>> 1. Scala AlexNet sample code calls Symbol.Convolution:
>>>>>>
>>>>>> https://github.com/apache/incubator-mxnet/blob/master/
>>>>>> scala-package/examples/src/main/scala/ml/dmlc/mxnetexamples/visualization/
>>>>>> AlexNet.scala#L30
>>>>>>
>>>>>> 2. Symbol.scala does not contain the string "Convolution"
>>>>>>
>>>>>> https://github.com/apache/incubator-mxnet/blob/master/
>>>>>> scala-package/core/src/main/scala/ml/dmlc/mxnet/Symbol.scala#L982
>>>>>>
>>>>>> Question: where/how is Symbol.Convolution defined?
>>>>>>
>>>>>> On Wed, Oct 18, 2017 at 4:10 PM, TongKe Xue <tkxue@tkxue.org>
wrote:
>>>>>> > Hi,
>>>>>> >
>>>>>> > I am reading: https://mxnet.incubator.apache.org/api/scala/symbol.html
>>>>>> >
>>>>>> > I see Symbol.Variable, Symbol.Convolution
>>>>>> >
>>>>>> > When I look at Symbol.scala, I see Symbol.Variable at:
>>>>>> > https://github.com/apache/incubator-mxnet/blob/master/
>>>>>> scala-package/core/src/main/scala/ml/dmlc/mxnet/Symbol.scala#L982
>>>>>> >
>>>>>> > However, I can't find where Convolution, SoftMax, FullyConnected,
...
>>>>>> > are defined.
>>>>>> >
>>>>>> > Where are these Symbols defined?
>>>>>> >
>>>>>> > (I have also tried: grep "Convolution" . -R | grep scala | grep
def --
>>>>>> > but found nothing).
>>>>>> >
>>>>>> > Thanks,
>>>>>> > --TongKe
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Rahul Huilgol
>>>
>>>
>>>
>>> --
>>> Yizhi Liu
>>> DMLC member
>>> Technical Manager
>>> Qihoo 360 Inc, Shanghai, China
>
>
>
> --
> Yizhi Liu
> DMLC member
> Technical Manager
> Qihoo 360 Inc, Shanghai, China



-- 
Yizhi Liu
DMLC member
Technical Manager
Qihoo 360 Inc, Shanghai, China

Mime
View raw message