mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YiZhi Liu <eazhi....@gmail.com>
Subject Re: Clojure Package
Date Sat, 09 Jun 2018 00:38:24 GMT
Carin,

Thank you for the great work. I'll do the review. As I have no
expertise in Clojure, it will really help to have people from Clojure
community reviewing it as well :)

2018-06-08 14:22 GMT-07:00 Carin Meier <carinmeier@gmail.com>:
> A couple of questions came up in regard to the PR and the current test
> suite state as well as the best way to review the PR since it is a new
> language binding.
>
> In regards to the Clojure test suite, most of the Scala test suite has been
> ported over with the goal of having comparable coverage. I can go ahead a
> put in a coverage tool to make that it a bit more transparent.
>
> For reviewing, I have a couple people in the Clojure community that are
> interested in collaborating in this project and I can ask them for help in
> reviewing the PR in some way if that is helpful too.
>
> I'm also open to other suggestions.
>
> Thanks,
> Carin
>
> On Fri, Jun 8, 2018 at 4:06 PM, Carin Meier <carinmeier@gmail.com> wrote:
>
>> Here is the PR https://github.com/apache/incubator-mxnet/pull/11205
>>
>> I've ported in the content from the external github repo (
>> https://github.com/gigasquid/clojure-mxnet), that has been tested by some
>> of the Clojure community, into the contrib directory.
>>
>> There is still lots more to do in relation to adding tests, benchmarks,
>> and increasing stability, but I thought this might be a good point to bring
>> it in initially so that the other work can be reviewed in smaller chunks.
>> I also would like to get other people involved in making it better, so I
>> thought that having the base package in there would be a good starting
>> point for collaboration.
>>
>> Feedback welcome.
>>
>> - Carin
>>
>>
>> On Tue, Jun 5, 2018 at 11:40 AM, Carin Meier <carinmeier@gmail.com> wrote:
>>
>>> Thanks everyone. I'll work on getting together a PR with your feedback
>>> and post it here.
>>>
>>> On Tue, Jun 5, 2018 at 4:05 AM, Chen HY <chenhy12345@gmail.com> wrote:
>>>
>>>> I would suggest using code generators in case upstream library adding the
>>>> functions for arrays.
>>>> It seems that cpp binding is using a code generator and works fine.
>>>>
>>>> 2018-06-05 7:59 GMT+01:00 Naveen Swamy <mnnaveen@gmail.com>:
>>>>
>>>> > ~/mxnet/contrib/clojure-package good place for the code.
>>>> >
>>>> > the package name org.apache.mxnet.contrib.clojure ? do you need mxnet
>>>> > again?
>>>> >
>>>> > I forgot to request to run some benchmarks and document. One of the
>>>> reasons
>>>> > users use MXNet is because of its performance and we want to ensure
>>>> that we
>>>> > maintain it across language bindings.
>>>> >
>>>> > Also invite your other clojure programmer buddies to the party :)
>>>> >
>>>> > Thanks, Naveen
>>>> >
>>>> >
>>>> > On Mon, Jun 4, 2018 at 1:55 PM, Carin Meier <carinmeier@gmail.com>
>>>> wrote:
>>>> >
>>>> > > Oh right. That's not a problem, I wonder if something like
>>>> > >
>>>> > > org.apache.mxnet.contrib/clojure-mxnet
>>>> > >
>>>> > > would work?
>>>> > >
>>>> > > If this seems like it is the right direction, we could work out
the
>>>> > details
>>>> > > in a PR.
>>>> > >
>>>> > >
>>>> > > On Mon, Jun 4, 2018 at 4:44 PM, Naveen Swamy <mnnaveen@gmail.com>
>>>> wrote:
>>>> > >
>>>> > > > I agree with your assessment that we shouldn't need the user
to
>>>> change
>>>> > > > their code. I am not sure if we can release under
>>>> > > org.apache.clojure-mxnet
>>>> > > > we might have to stick with our primary group id org.apache.mxnet
>>>> and
>>>> > may
>>>> > > > be create a sub-package under it? any creative ideas?
>>>> > > >
>>>> > > > On Mon, Jun 4, 2018 at 1:29 PM, Carin Meier <carinmeier@gmail.com>
>>>> > > wrote:
>>>> > > >
>>>> > > > > Thanks for the feedback everyone.
>>>> > > > >
>>>> > > > > I agree on the contrib option. I think it's a great path
forward
>>>> and
>>>> > > > would
>>>> > > > > allow it time for feedback, contribution by others, and
>>>> > stabilization.
>>>> > > > >
>>>> > > > > If I'm understanding correctly, that would mean putting
the
>>>> source
>>>> > code
>>>> > > > in:
>>>> > > > > ~/mxnet/contrib/clojure-package
>>>> > > > >
>>>> > > > > and having the artifact jar named
>>>> > > > > `org.apache.contrib.clojure-mxnet/clojure-mxnet`
>>>> > > > >
>>>> > > > > I would recommend not having the individual namespaces
of the
>>>> files
>>>> > > have
>>>> > > > > contrib embedded in them, so that if it graduates, users
won't
>>>> have
>>>> > to
>>>> > > > > change their code, only the dependency.
>>>> > > > >
>>>> > > > > Please let me know if this is correct or if there are
any other
>>>> > ideas.
>>>> > > > >
>>>> > > > > - Carin
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > >
>>>> > > > > On Mon, Jun 4, 2018 at 4:03 PM, Naveen Swamy <mnnaveen@gmail.com
>>>> >
>>>> > > wrote:
>>>> > > > >
>>>> > > > > > I think that's a great idea to bring in under contrib
and we
>>>> can
>>>> > also
>>>> > > > get
>>>> > > > > > user feedback
>>>> > > > > >
>>>> > > > > > > On Jun 4, 2018, at 12:44 PM, sandeep krishnamurthy
<
>>>> > > > > > sandeep.krishna98@gmail.com> wrote:
>>>> > > > > > >
>>>> > > > > > > Hi Carin,
>>>> > > > > > >
>>>> > > > > > > This is a commendable work. Thanks a lot for
all the hard and
>>>> > smart
>>>> > > > > work
>>>> > > > > > > you have put behind this :-) I think this will
be a great
>>>> value
>>>> > > > > addition.
>>>> > > > > > >
>>>> > > > > > > If people are not sure about usage, can I suggest
this
>>>> awesome
>>>> > work
>>>> > > > to
>>>> > > > > be
>>>> > > > > > > brought in "contrib" package? Invite and build
the community
>>>> > around
>>>> > > > > > > Clojure, stabilize and increase the coverage,
and based on
>>>> usage
>>>> > > and
>>>> > > > > > > development, graduate it to main stable support
from contrib.
>>>> > > > > > >
>>>> > > > > > > Suggestions and thoughts?
>>>> > > > > > >
>>>> > > > > > > Best,
>>>> > > > > > > Sandeep
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > On Mon, Jun 4, 2018 at 12:27 PM, Ivan Serdyuk
<
>>>> > > > > > local.tourist.kiev@gmail.com>
>>>> > > > > > > wrote:
>>>> > > > > > >
>>>> > > > > > >> Hello everyone.
>>>> > > > > > >>
>>>> > > > > > >> A small comment, about Scala API: main
commiters are hardly
>>>> > > > available,
>>>> > > > > > as
>>>> > > > > > >> for today.
>>>> > > > > > >>
>>>> > > > > > >> As for Clojure - I might suggest that it
might be possible
>>>> to
>>>> > > > enlight
>>>> > > > > > >> future work, for that package, for Clojure
developers.
>>>> > > > > > >>
>>>> > > > > > >>> On Mon, Jun 4, 2018 at 8:50 PM, Naveen
Swamy <
>>>> > mnnaveen@gmail.com
>>>> > > >
>>>> > > > > > wrote:
>>>> > > > > > >>>
>>>> > > > > > >>> Hi Carin,
>>>> > > > > > >>>
>>>> > > > > > >>> This is great effort and very nicely
documented. When I
>>>> > surveyed
>>>> > > at
>>>> > > > > my
>>>> > > > > > >> day
>>>> > > > > > >>> job on the interest for Scala packages
there were a few
>>>> people
>>>> > > who
>>>> > > > > > >>> specifically were interested in the
Clojure packages, I
>>>> think
>>>> > > those
>>>> > > > > are
>>>> > > > > > >>> people who might have tried and understand
the complexity
>>>> of
>>>> > > making
>>>> > > > > > Scala
>>>> > > > > > >>> work in Clojure.
>>>> > > > > > >>>
>>>> > > > > > >>> I am in support of a MXNet-Clojure
package, I request you
>>>> to
>>>> > add
>>>> > > > more
>>>> > > > > > >> unit
>>>> > > > > > >>> tests and integration tests that can
be ported to CI.
>>>> > > > > > >>>
>>>> > > > > > >>> Thanks for your efforts.
>>>> > > > > > >>>
>>>> > > > > > >>> -Naveen
>>>> > > > > > >>>
>>>> > > > > > >>>
>>>> > > > > > >>>
>>>> > > > > > >>> On Fri, Jun 1, 2018 at 6:16 PM, Carin
Meier <
>>>> > > carinmeier@gmail.com>
>>>> > > > > > >> wrote:
>>>> > > > > > >>>
>>>> > > > > > >>>> It is always a good thing to consider
the cost with the
>>>> > benefit.
>>>> > > > > I'll
>>>> > > > > > >> do
>>>> > > > > > >>> my
>>>> > > > > > >>>> best to explain what I see the
tradeoffs to be.
>>>> > > > > > >>>>
>>>> > > > > > >>>> First, I wanted to clarify that
it took significant
>>>> > development
>>>> > > > > effort
>>>> > > > > > >> to
>>>> > > > > > >>>> get the Clojure package and the
interop working properly
>>>> > despite
>>>> > > > my
>>>> > > > > > >>> simple
>>>> > > > > > >>>> looking design on the confluence
page :)
>>>> > > > > > >>>>
>>>> > > > > > >>>> One of the advantages of MXNet
over its competitors is its
>>>> > many
>>>> > > > > > >> language
>>>> > > > > > >>>> support. The Clojure package would
only increase the
>>>> value of
>>>> > > this
>>>> > > > > > >>>> proposition and bring new users
and growth into the
>>>> community.
>>>> > > > > > >>>> However, there is a cost associated
with adding this
>>>> language
>>>> > > > > support
>>>> > > > > > >> as
>>>> > > > > > >>>> you pointed out.
>>>> > > > > > >>>>
>>>> > > > > > >>>> Since the Clojure package right
now is only reliant on the
>>>> > Scala
>>>> > > > > jars
>>>> > > > > > >>> from
>>>> > > > > > >>>> Maven, it can exist outside the
main project as an
>>>> independent
>>>> > > > repo
>>>> > > > > > >> but I
>>>> > > > > > >>>> think that would lessen the growth
benefit both to the
>>>> Clojure
>>>> > > > > > >> community
>>>> > > > > > >>>> and to the MXNet community to not
be included as a first
>>>> class
>>>> > > > > > >> language.
>>>> > > > > > >>>>
>>>> > > > > > >>>> I believe having first class Clojure
support in MXNet is
>>>> > > valuable,
>>>> > > > > but
>>>> > > > > > >>> the
>>>> > > > > > >>>> cost of that support is up to the
community to decide.
>>>> > > > > > >>>>
>>>> > > > > > >>>> Is there a process for considering
a new package in MXNet?
>>>> > > > > > >>>>
>>>> > > > > > >>>> - Carin
>>>> > > > > > >>>>
>>>> > > > > > >>>>> On Fri, Jun 1, 2018 at 5:51
PM, Chen HY <
>>>> > chenhy12345@gmail.com
>>>> > > >
>>>> > > > > > wrote:
>>>> > > > > > >>>>>
>>>> > > > > > >>>>> Have checked the issue and
the confluence page, but still
>>>> > > > curious.
>>>> > > > > > >>>>> Clojure and Scala are both
JVM based languages.
>>>> > > > > > >>>>> They, as well as many JVM based
languages, can share
>>>> their
>>>> > > class
>>>> > > > > and
>>>> > > > > > >>>> method
>>>> > > > > > >>>>> at a certain level.
>>>> > > > > > >>>>> Why should the community maintain
two APIs for two
>>>> languages
>>>> > > with
>>>> > > > > can
>>>> > > > > > >>>> share
>>>> > > > > > >>>>> their packages with almost
zero effort?
>>>> > > > > > >>>>>
>>>> > > > > > >>>>>
>>>> > > > > > >>>>> 2018-06-01 21:58 GMT+01:00
Carin Meier <
>>>> carinmeier@gmail.com
>>>> > >:
>>>> > > > > > >>>>>
>>>> > > > > > >>>>>> Hi all,
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> I've been working on a
Clojure package for MXNet. Since
>>>> > > Clojure
>>>> > > > is
>>>> > > > > > >> a
>>>> > > > > > >>>> JVM
>>>> > > > > > >>>>>> language, the package leverages
the great work of the
>>>> > existing
>>>> > > > > > >> Scala
>>>> > > > > > >>>>>> package.
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> I would appreciate any
feedback and testing.
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> Here is the original issue:
>>>> > > > > > >>>>>> https://github.com/apache/incubator-mxnet/issues/8971
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> Architecture & Design:
>>>> > > > > > >>>>>> https://cwiki.apache.org/confluence/display/MXNET/
>>>> > > MXNet+Clojure
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> and the github repo for
rapid testing and issue fixing
>>>> > before
>>>> > > of
>>>> > > > > > >>>> opening
>>>> > > > > > >>>>> an
>>>> > > > > > >>>>>> official PR https://github.com/gigasquid/clojure-mxnet
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> I'm also active in the
slack channel so feel free to
>>>> ping me
>>>> > > > > there.
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>> Thanks,
>>>> > > > > > >>>>>> Carin Meier
>>>> > > > > > >>>>>>
>>>> > > > > > >>>>>
>>>> > > > > > >>>>>
>>>> > > > > > >>>>>
>>>> > > > > > >>>>> --
>>>> > > > > > >>>>> Chen Hanyang 陈涵洋
>>>> > > > > > >>>>> Software School Fudan University
>>>> > > > > > >>>>> +86-138-1881-7745
>>>> > > > > > >>>>>
>>>> > > > > > >>>>
>>>> > > > > > >>>
>>>> > > > > > >>
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > --
>>>> > > > > > > Sandeep Krishnamurthy
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Chen Hanyang 陈涵洋
>>>> Software School Fudan University
>>>> +86-138-1881-7745
>>>>
>>>
>>>
>>



-- 
Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada

Mime
View raw message