mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naveen Swamy <mnnav...@gmail.com>
Subject Re: Clojure Package
Date Sun, 10 Jun 2018 11:45:21 GMT
I have called for contribution within my organization to help with the
review, some of them have shown interest and hope to see them help with the
review. Carin, I learnt you are popular to have given great talks on
Clojure.

Could I please ask you document what functionality is working and what
parts of it need contribution.

In general I want to request a few things:
Good API documentation,
Examples, turn examples into integration tests.
Tests, more tests
Everything testable on CI.

In the long term, We have to think how to maintain API compatibility and
tests across language bindings.

Thanks, Naveen


On Sat, Jun 9, 2018 at 2:38 AM, YiZhi Liu <eazhi.liu@gmail.com> wrote:

> Carin,
>
> Thank you for the great work. I'll do the review. As I have no
> expertise in Clojure, it will really help to have people from Clojure
> community reviewing it as well :)
>
> 2018-06-08 14:22 GMT-07:00 Carin Meier <carinmeier@gmail.com>:
> > A couple of questions came up in regard to the PR and the current test
> > suite state as well as the best way to review the PR since it is a new
> > language binding.
> >
> > In regards to the Clojure test suite, most of the Scala test suite has
> been
> > ported over with the goal of having comparable coverage. I can go ahead a
> > put in a coverage tool to make that it a bit more transparent.
> >
> > For reviewing, I have a couple people in the Clojure community that are
> > interested in collaborating in this project and I can ask them for help
> in
> > reviewing the PR in some way if that is helpful too.
> >
> > I'm also open to other suggestions.
> >
> > Thanks,
> > Carin
> >
> > On Fri, Jun 8, 2018 at 4:06 PM, Carin Meier <carinmeier@gmail.com>
> wrote:
> >
> >> Here is the PR https://github.com/apache/incubator-mxnet/pull/11205
> >>
> >> I've ported in the content from the external github repo (
> >> https://github.com/gigasquid/clojure-mxnet), that has been tested by
> some
> >> of the Clojure community, into the contrib directory.
> >>
> >> There is still lots more to do in relation to adding tests, benchmarks,
> >> and increasing stability, but I thought this might be a good point to
> bring
> >> it in initially so that the other work can be reviewed in smaller
> chunks.
> >> I also would like to get other people involved in making it better, so I
> >> thought that having the base package in there would be a good starting
> >> point for collaboration.
> >>
> >> Feedback welcome.
> >>
> >> - Carin
> >>
> >>
> >> On Tue, Jun 5, 2018 at 11:40 AM, Carin Meier <carinmeier@gmail.com>
> wrote:
> >>
> >>> Thanks everyone. I'll work on getting together a PR with your feedback
> >>> and post it here.
> >>>
> >>> On Tue, Jun 5, 2018 at 4:05 AM, Chen HY <chenhy12345@gmail.com> wrote:
> >>>
> >>>> I would suggest using code generators in case upstream library adding
> the
> >>>> functions for arrays.
> >>>> It seems that cpp binding is using a code generator and works fine.
> >>>>
> >>>> 2018-06-05 7:59 GMT+01:00 Naveen Swamy <mnnaveen@gmail.com>:
> >>>>
> >>>> > ~/mxnet/contrib/clojure-package good place for the code.
> >>>> >
> >>>> > the package name org.apache.mxnet.contrib.clojure ? do you need
> mxnet
> >>>> > again?
> >>>> >
> >>>> > I forgot to request to run some benchmarks and document. One of
the
> >>>> reasons
> >>>> > users use MXNet is because of its performance and we want to ensure
> >>>> that we
> >>>> > maintain it across language bindings.
> >>>> >
> >>>> > Also invite your other clojure programmer buddies to the party
:)
> >>>> >
> >>>> > Thanks, Naveen
> >>>> >
> >>>> >
> >>>> > On Mon, Jun 4, 2018 at 1:55 PM, Carin Meier <carinmeier@gmail.com>
> >>>> wrote:
> >>>> >
> >>>> > > Oh right. That's not a problem, I wonder if something like
> >>>> > >
> >>>> > > org.apache.mxnet.contrib/clojure-mxnet
> >>>> > >
> >>>> > > would work?
> >>>> > >
> >>>> > > If this seems like it is the right direction, we could work
out
> the
> >>>> > details
> >>>> > > in a PR.
> >>>> > >
> >>>> > >
> >>>> > > On Mon, Jun 4, 2018 at 4:44 PM, Naveen Swamy <mnnaveen@gmail.com>
> >>>> wrote:
> >>>> > >
> >>>> > > > I agree with your assessment that we shouldn't need the
user to
> >>>> change
> >>>> > > > their code. I am not sure if we can release under
> >>>> > > org.apache.clojure-mxnet
> >>>> > > > we might have to stick with our primary group id
> org.apache.mxnet
> >>>> and
> >>>> > may
> >>>> > > > be create a sub-package under it? any creative ideas?
> >>>> > > >
> >>>> > > > On Mon, Jun 4, 2018 at 1:29 PM, Carin Meier <
> carinmeier@gmail.com>
> >>>> > > wrote:
> >>>> > > >
> >>>> > > > > Thanks for the feedback everyone.
> >>>> > > > >
> >>>> > > > > I agree on the contrib option. I think it's a great
path
> forward
> >>>> and
> >>>> > > > would
> >>>> > > > > allow it time for feedback, contribution by others,
and
> >>>> > stabilization.
> >>>> > > > >
> >>>> > > > > If I'm understanding correctly, that would mean
putting the
> >>>> source
> >>>> > code
> >>>> > > > in:
> >>>> > > > > ~/mxnet/contrib/clojure-package
> >>>> > > > >
> >>>> > > > > and having the artifact jar named
> >>>> > > > > `org.apache.contrib.clojure-mxnet/clojure-mxnet`
> >>>> > > > >
> >>>> > > > > I would recommend not having the individual namespaces
of the
> >>>> files
> >>>> > > have
> >>>> > > > > contrib embedded in them, so that if it graduates,
users won't
> >>>> have
> >>>> > to
> >>>> > > > > change their code, only the dependency.
> >>>> > > > >
> >>>> > > > > Please let me know if this is correct or if there
are any
> other
> >>>> > ideas.
> >>>> > > > >
> >>>> > > > > - Carin
> >>>> > > > >
> >>>> > > > >
> >>>> > > > >
> >>>> > > > >
> >>>> > > > > On Mon, Jun 4, 2018 at 4:03 PM, Naveen Swamy <
> mnnaveen@gmail.com
> >>>> >
> >>>> > > wrote:
> >>>> > > > >
> >>>> > > > > > I think that's a great idea to bring in under
contrib and we
> >>>> can
> >>>> > also
> >>>> > > > get
> >>>> > > > > > user feedback
> >>>> > > > > >
> >>>> > > > > > > On Jun 4, 2018, at 12:44 PM, sandeep krishnamurthy
<
> >>>> > > > > > sandeep.krishna98@gmail.com> wrote:
> >>>> > > > > > >
> >>>> > > > > > > Hi Carin,
> >>>> > > > > > >
> >>>> > > > > > > This is a commendable work. Thanks a lot
for all the hard
> and
> >>>> > smart
> >>>> > > > > work
> >>>> > > > > > > you have put behind this :-) I think this
will be a great
> >>>> value
> >>>> > > > > addition.
> >>>> > > > > > >
> >>>> > > > > > > If people are not sure about usage, can
I suggest this
> >>>> awesome
> >>>> > work
> >>>> > > > to
> >>>> > > > > be
> >>>> > > > > > > brought in "contrib" package? Invite and
build the
> community
> >>>> > around
> >>>> > > > > > > Clojure, stabilize and increase the coverage,
and based on
> >>>> usage
> >>>> > > and
> >>>> > > > > > > development, graduate it to main stable
support from
> contrib.
> >>>> > > > > > >
> >>>> > > > > > > Suggestions and thoughts?
> >>>> > > > > > >
> >>>> > > > > > > Best,
> >>>> > > > > > > Sandeep
> >>>> > > > > > >
> >>>> > > > > > >
> >>>> > > > > > >
> >>>> > > > > > > On Mon, Jun 4, 2018 at 12:27 PM, Ivan
Serdyuk <
> >>>> > > > > > local.tourist.kiev@gmail.com>
> >>>> > > > > > > wrote:
> >>>> > > > > > >
> >>>> > > > > > >> Hello everyone.
> >>>> > > > > > >>
> >>>> > > > > > >> A small comment, about Scala API:
main commiters are
> hardly
> >>>> > > > available,
> >>>> > > > > > as
> >>>> > > > > > >> for today.
> >>>> > > > > > >>
> >>>> > > > > > >> As for Clojure - I might suggest that
it might be
> possible
> >>>> to
> >>>> > > > enlight
> >>>> > > > > > >> future work, for that package, for
Clojure developers.
> >>>> > > > > > >>
> >>>> > > > > > >>> On Mon, Jun 4, 2018 at 8:50 PM,
Naveen Swamy <
> >>>> > mnnaveen@gmail.com
> >>>> > > >
> >>>> > > > > > wrote:
> >>>> > > > > > >>>
> >>>> > > > > > >>> Hi Carin,
> >>>> > > > > > >>>
> >>>> > > > > > >>> This is great effort and very
nicely documented. When I
> >>>> > surveyed
> >>>> > > at
> >>>> > > > > my
> >>>> > > > > > >> day
> >>>> > > > > > >>> job on the interest for Scala
packages there were a few
> >>>> people
> >>>> > > who
> >>>> > > > > > >>> specifically were interested in
the Clojure packages, I
> >>>> think
> >>>> > > those
> >>>> > > > > are
> >>>> > > > > > >>> people who might have tried and
understand the
> complexity
> >>>> of
> >>>> > > making
> >>>> > > > > > Scala
> >>>> > > > > > >>> work in Clojure.
> >>>> > > > > > >>>
> >>>> > > > > > >>> I am in support of a MXNet-Clojure
package, I request
> you
> >>>> to
> >>>> > add
> >>>> > > > more
> >>>> > > > > > >> unit
> >>>> > > > > > >>> tests and integration tests that
can be ported to CI.
> >>>> > > > > > >>>
> >>>> > > > > > >>> Thanks for your efforts.
> >>>> > > > > > >>>
> >>>> > > > > > >>> -Naveen
> >>>> > > > > > >>>
> >>>> > > > > > >>>
> >>>> > > > > > >>>
> >>>> > > > > > >>> On Fri, Jun 1, 2018 at 6:16 PM,
Carin Meier <
> >>>> > > carinmeier@gmail.com>
> >>>> > > > > > >> wrote:
> >>>> > > > > > >>>
> >>>> > > > > > >>>> It is always a good thing
to consider the cost with the
> >>>> > benefit.
> >>>> > > > > I'll
> >>>> > > > > > >> do
> >>>> > > > > > >>> my
> >>>> > > > > > >>>> best to explain what I see
the tradeoffs to be.
> >>>> > > > > > >>>>
> >>>> > > > > > >>>> First, I wanted to clarify
that it took significant
> >>>> > development
> >>>> > > > > effort
> >>>> > > > > > >> to
> >>>> > > > > > >>>> get the Clojure package and
the interop working
> properly
> >>>> > despite
> >>>> > > > my
> >>>> > > > > > >>> simple
> >>>> > > > > > >>>> looking design on the confluence
page :)
> >>>> > > > > > >>>>
> >>>> > > > > > >>>> One of the advantages of MXNet
over its competitors is
> its
> >>>> > many
> >>>> > > > > > >> language
> >>>> > > > > > >>>> support. The Clojure package
would only increase the
> >>>> value of
> >>>> > > this
> >>>> > > > > > >>>> proposition and bring new
users and growth into the
> >>>> community.
> >>>> > > > > > >>>> However, there is a cost associated
with adding this
> >>>> language
> >>>> > > > > support
> >>>> > > > > > >> as
> >>>> > > > > > >>>> you pointed out.
> >>>> > > > > > >>>>
> >>>> > > > > > >>>> Since the Clojure package
right now is only reliant on
> the
> >>>> > Scala
> >>>> > > > > jars
> >>>> > > > > > >>> from
> >>>> > > > > > >>>> Maven, it can exist outside
the main project as an
> >>>> independent
> >>>> > > > repo
> >>>> > > > > > >> but I
> >>>> > > > > > >>>> think that would lessen the
growth benefit both to the
> >>>> Clojure
> >>>> > > > > > >> community
> >>>> > > > > > >>>> and to the MXNet community
to not be included as a
> first
> >>>> class
> >>>> > > > > > >> language.
> >>>> > > > > > >>>>
> >>>> > > > > > >>>> I believe having first class
Clojure support in MXNet
> is
> >>>> > > valuable,
> >>>> > > > > but
> >>>> > > > > > >>> the
> >>>> > > > > > >>>> cost of that support is up
to the community to decide.
> >>>> > > > > > >>>>
> >>>> > > > > > >>>> Is there a process for considering
a new package in
> MXNet?
> >>>> > > > > > >>>>
> >>>> > > > > > >>>> - Carin
> >>>> > > > > > >>>>
> >>>> > > > > > >>>>> On Fri, Jun 1, 2018 at
5:51 PM, Chen HY <
> >>>> > chenhy12345@gmail.com
> >>>> > > >
> >>>> > > > > > wrote:
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>> Have checked the issue
and the confluence page, but
> still
> >>>> > > > curious.
> >>>> > > > > > >>>>> Clojure and Scala are
both JVM based languages.
> >>>> > > > > > >>>>> They, as well as many
JVM based languages, can share
> >>>> their
> >>>> > > class
> >>>> > > > > and
> >>>> > > > > > >>>> method
> >>>> > > > > > >>>>> at a certain level.
> >>>> > > > > > >>>>> Why should the community
maintain two APIs for two
> >>>> languages
> >>>> > > with
> >>>> > > > > can
> >>>> > > > > > >>>> share
> >>>> > > > > > >>>>> their packages with almost
zero effort?
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>> 2018-06-01 21:58 GMT+01:00
Carin Meier <
> >>>> carinmeier@gmail.com
> >>>> > >:
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>>> Hi all,
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> I've been working
on a Clojure package for MXNet.
> Since
> >>>> > > Clojure
> >>>> > > > is
> >>>> > > > > > >> a
> >>>> > > > > > >>>> JVM
> >>>> > > > > > >>>>>> language, the package
leverages the great work of the
> >>>> > existing
> >>>> > > > > > >> Scala
> >>>> > > > > > >>>>>> package.
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> I would appreciate
any feedback and testing.
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> Here is the original
issue:
> >>>> > > > > > >>>>>> https://github.com/apache/
> incubator-mxnet/issues/8971
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> Architecture &
Design:
> >>>> > > > > > >>>>>> https://cwiki.apache.org/confluence/display/MXNET/
> >>>> > > MXNet+Clojure
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> and the github repo
for rapid testing and issue
> fixing
> >>>> > before
> >>>> > > of
> >>>> > > > > > >>>> opening
> >>>> > > > > > >>>>> an
> >>>> > > > > > >>>>>> official PR https://github.com/gigasquid/
> clojure-mxnet
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> I'm also active in
the slack channel so feel free to
> >>>> ping me
> >>>> > > > > there.
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>> Thanks,
> >>>> > > > > > >>>>>> Carin Meier
> >>>> > > > > > >>>>>>
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>> --
> >>>> > > > > > >>>>> Chen Hanyang 陈涵洋
> >>>> > > > > > >>>>> Software School Fudan
University
> >>>> > > > > > >>>>> +86-138-1881-7745
> >>>> > > > > > >>>>>
> >>>> > > > > > >>>>
> >>>> > > > > > >>>
> >>>> > > > > > >>
> >>>> > > > > > >
> >>>> > > > > > >
> >>>> > > > > > >
> >>>> > > > > > > --
> >>>> > > > > > > Sandeep Krishnamurthy
> >>>> > > > > >
> >>>> > > > >
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Chen Hanyang 陈涵洋
> >>>> Software School Fudan University
> >>>> +86-138-1881-7745
> >>>>
> >>>
> >>>
> >>
>
>
>
> --
> Yizhi Liu
> DMLC member
> Amazon Web Services
> Vancouver, Canada
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message