mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naveen Swamy <mnnav...@gmail.com>
Subject Re: Clojure Package
Date Sun, 10 Jun 2018 13:19:33 GMT
this is fantastic, thanks for being a champ. I am glad to have you join us
:). Yes, we would love to make it work and be available for our users, you
can make features available iteratively. I understand you alone can't have
everything in one go, I am hoping more contributors will join the effort.

My guess is Chen is recommending generation of Clojure code from Scala.

On Sun, Jun 10, 2018 at 2:55 PM, Carin Meier <carinmeier@gmail.com> wrote:

> Naveen,
>
> Thanks. I appreciate your help. I acknowledge there are some challenges in
> trying to bring in a new language package, but I think if we can make it
> work, it will be the best way for it to grow along with the MXNet project
> and be a good fit with its goals and ecosystem long term.
>
> I added a confluence page of the general state of what is working and needs
> help.
> https://cwiki.apache.org/confluence/display/MXNET/
> Clojure+Package+Contribution+Needs.
> Feel free to edit.
>
> Chen,
>
> I added the code generator use on the page as well. I don't fully
> understand what you are proposing, but I would like to. If you could
> provide some more details to help me, that would be great.
>
> - Carin
>
>
> On Sun, Jun 10, 2018 at 8:00 AM, Chen HY <chenhy12345@gmail.com> wrote:
>
> > I personally think that is the reason the clojure package should use more
> > code generator.
> > It makes easier to build everything align with scala package.
> >
> > 2018-06-10 12:45 GMT+01:00 Naveen Swamy <mnnaveen@gmail.com>:
> >
> > > I have called for contribution within my organization to help with the
> > > review, some of them have shown interest and hope to see them help with
> > the
> > > review. Carin, I learnt you are popular to have given great talks on
> > > Clojure.
> > >
> > > Could I please ask you document what functionality is working and what
> > > parts of it need contribution.
> > >
> > > In general I want to request a few things:
> > > Good API documentation,
> > > Examples, turn examples into integration tests.
> > > Tests, more tests
> > > Everything testable on CI.
> > >
> > > In the long term, We have to think how to maintain API compatibility
> and
> > > tests across language bindings.
> > >
> > > Thanks, Naveen
> > >
> > >
> > > On Sat, Jun 9, 2018 at 2:38 AM, YiZhi Liu <eazhi.liu@gmail.com> wrote:
> > >
> > > > Carin,
> > > >
> > > > Thank you for the great work. I'll do the review. As I have no
> > > > expertise in Clojure, it will really help to have people from Clojure
> > > > community reviewing it as well :)
> > > >
> > > > 2018-06-08 14:22 GMT-07:00 Carin Meier <carinmeier@gmail.com>:
> > > > > A couple of questions came up in regard to the PR and the current
> > test
> > > > > suite state as well as the best way to review the PR since it is
a
> > new
> > > > > language binding.
> > > > >
> > > > > In regards to the Clojure test suite, most of the Scala test suite
> > has
> > > > been
> > > > > ported over with the goal of having comparable coverage. I can go
> > > ahead a
> > > > > put in a coverage tool to make that it a bit more transparent.
> > > > >
> > > > > For reviewing, I have a couple people in the Clojure community that
> > are
> > > > > interested in collaborating in this project and I can ask them for
> > help
> > > > in
> > > > > reviewing the PR in some way if that is helpful too.
> > > > >
> > > > > I'm also open to other suggestions.
> > > > >
> > > > > Thanks,
> > > > > Carin
> > > > >
> > > > > On Fri, Jun 8, 2018 at 4:06 PM, Carin Meier <carinmeier@gmail.com>
> > > > wrote:
> > > > >
> > > > >> Here is the PR https://github.com/apache/
> incubator-mxnet/pull/11205
> > > > >>
> > > > >> I've ported in the content from the external github repo (
> > > > >> https://github.com/gigasquid/clojure-mxnet), that has been tested
> > by
> > > > some
> > > > >> of the Clojure community, into the contrib directory.
> > > > >>
> > > > >> There is still lots more to do in relation to adding tests,
> > > benchmarks,
> > > > >> and increasing stability, but I thought this might be a good
point
> > to
> > > > bring
> > > > >> it in initially so that the other work can be reviewed in smaller
> > > > chunks.
> > > > >> I also would like to get other people involved in making it
> better,
> > > so I
> > > > >> thought that having the base package in there would be a good
> > starting
> > > > >> point for collaboration.
> > > > >>
> > > > >> Feedback welcome.
> > > > >>
> > > > >> - Carin
> > > > >>
> > > > >>
> > > > >> On Tue, Jun 5, 2018 at 11:40 AM, Carin Meier <
> carinmeier@gmail.com>
> > > > wrote:
> > > > >>
> > > > >>> Thanks everyone. I'll work on getting together a PR with
your
> > > feedback
> > > > >>> and post it here.
> > > > >>>
> > > > >>> On Tue, Jun 5, 2018 at 4:05 AM, Chen HY <chenhy12345@gmail.com>
> > > wrote:
> > > > >>>
> > > > >>>> I would suggest using code generators in case upstream
library
> > > adding
> > > > the
> > > > >>>> functions for arrays.
> > > > >>>> It seems that cpp binding is using a code generator and
works
> > fine.
> > > > >>>>
> > > > >>>> 2018-06-05 7:59 GMT+01:00 Naveen Swamy <mnnaveen@gmail.com>:
> > > > >>>>
> > > > >>>> > ~/mxnet/contrib/clojure-package good place for the
code.
> > > > >>>> >
> > > > >>>> > the package name org.apache.mxnet.contrib.clojure
? do you
> need
> > > > mxnet
> > > > >>>> > again?
> > > > >>>> >
> > > > >>>> > I forgot to request to run some benchmarks and document.
One
> of
> > > the
> > > > >>>> reasons
> > > > >>>> > users use MXNet is because of its performance and
we want to
> > > ensure
> > > > >>>> that we
> > > > >>>> > maintain it across language bindings.
> > > > >>>> >
> > > > >>>> > Also invite your other clojure programmer buddies
to the party
> > :)
> > > > >>>> >
> > > > >>>> > Thanks, Naveen
> > > > >>>> >
> > > > >>>> >
> > > > >>>> > On Mon, Jun 4, 2018 at 1:55 PM, Carin Meier <
> > carinmeier@gmail.com
> > > >
> > > > >>>> wrote:
> > > > >>>> >
> > > > >>>> > > Oh right. That's not a problem, I wonder if
something like
> > > > >>>> > >
> > > > >>>> > > org.apache.mxnet.contrib/clojure-mxnet
> > > > >>>> > >
> > > > >>>> > > would work?
> > > > >>>> > >
> > > > >>>> > > If this seems like it is the right direction,
we could work
> > out
> > > > the
> > > > >>>> > details
> > > > >>>> > > in a PR.
> > > > >>>> > >
> > > > >>>> > >
> > > > >>>> > > On Mon, Jun 4, 2018 at 4:44 PM, Naveen Swamy
<
> > > mnnaveen@gmail.com>
> > > > >>>> wrote:
> > > > >>>> > >
> > > > >>>> > > > I agree with your assessment that we shouldn't
need the
> user
> > > to
> > > > >>>> change
> > > > >>>> > > > their code. I am not sure if we can release
under
> > > > >>>> > > org.apache.clojure-mxnet
> > > > >>>> > > > we might have to stick with our primary
group id
> > > > org.apache.mxnet
> > > > >>>> and
> > > > >>>> > may
> > > > >>>> > > > be create a sub-package under it? any
creative ideas?
> > > > >>>> > > >
> > > > >>>> > > > On Mon, Jun 4, 2018 at 1:29 PM, Carin
Meier <
> > > > carinmeier@gmail.com>
> > > > >>>> > > wrote:
> > > > >>>> > > >
> > > > >>>> > > > > Thanks for the feedback everyone.
> > > > >>>> > > > >
> > > > >>>> > > > > I agree on the contrib option. I
think it's a great path
> > > > forward
> > > > >>>> and
> > > > >>>> > > > would
> > > > >>>> > > > > allow it time for feedback, contribution
by others, and
> > > > >>>> > stabilization.
> > > > >>>> > > > >
> > > > >>>> > > > > If I'm understanding correctly, that
would mean putting
> > the
> > > > >>>> source
> > > > >>>> > code
> > > > >>>> > > > in:
> > > > >>>> > > > > ~/mxnet/contrib/clojure-package
> > > > >>>> > > > >
> > > > >>>> > > > > and having the artifact jar named
> > > > >>>> > > > > `org.apache.contrib.clojure-mxnet/clojure-mxnet`
> > > > >>>> > > > >
> > > > >>>> > > > > I would recommend not having the
individual namespaces
> of
> > > the
> > > > >>>> files
> > > > >>>> > > have
> > > > >>>> > > > > contrib embedded in them, so that
if it graduates, users
> > > won't
> > > > >>>> have
> > > > >>>> > to
> > > > >>>> > > > > change their code, only the dependency.
> > > > >>>> > > > >
> > > > >>>> > > > > Please let me know if this is correct
or if there are
> any
> > > > other
> > > > >>>> > ideas.
> > > > >>>> > > > >
> > > > >>>> > > > > - Carin
> > > > >>>> > > > >
> > > > >>>> > > > >
> > > > >>>> > > > >
> > > > >>>> > > > >
> > > > >>>> > > > > On Mon, Jun 4, 2018 at 4:03 PM, Naveen
Swamy <
> > > > mnnaveen@gmail.com
> > > > >>>> >
> > > > >>>> > > wrote:
> > > > >>>> > > > >
> > > > >>>> > > > > > I think that's a great idea
to bring in under contrib
> > and
> > > we
> > > > >>>> can
> > > > >>>> > also
> > > > >>>> > > > get
> > > > >>>> > > > > > user feedback
> > > > >>>> > > > > >
> > > > >>>> > > > > > > On Jun 4, 2018, at 12:44
PM, sandeep krishnamurthy <
> > > > >>>> > > > > > sandeep.krishna98@gmail.com>
wrote:
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > Hi Carin,
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > This is a commendable work.
Thanks a lot for all the
> > > hard
> > > > and
> > > > >>>> > smart
> > > > >>>> > > > > work
> > > > >>>> > > > > > > you have put behind this
:-) I think this will be a
> > > great
> > > > >>>> value
> > > > >>>> > > > > addition.
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > If people are not sure
about usage, can I suggest
> this
> > > > >>>> awesome
> > > > >>>> > work
> > > > >>>> > > > to
> > > > >>>> > > > > be
> > > > >>>> > > > > > > brought in "contrib" package?
Invite and build the
> > > > community
> > > > >>>> > around
> > > > >>>> > > > > > > Clojure, stabilize and
increase the coverage, and
> > based
> > > on
> > > > >>>> usage
> > > > >>>> > > and
> > > > >>>> > > > > > > development, graduate it
to main stable support from
> > > > contrib.
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > Suggestions and thoughts?
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > Best,
> > > > >>>> > > > > > > Sandeep
> > > > >>>> > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > On Mon, Jun 4, 2018 at
12:27 PM, Ivan Serdyuk <
> > > > >>>> > > > > > local.tourist.kiev@gmail.com>
> > > > >>>> > > > > > > wrote:
> > > > >>>> > > > > > >
> > > > >>>> > > > > > >> Hello everyone.
> > > > >>>> > > > > > >>
> > > > >>>> > > > > > >> A small comment, about
Scala API: main commiters
> are
> > > > hardly
> > > > >>>> > > > available,
> > > > >>>> > > > > > as
> > > > >>>> > > > > > >> for today.
> > > > >>>> > > > > > >>
> > > > >>>> > > > > > >> As for Clojure - I
might suggest that it might be
> > > > possible
> > > > >>>> to
> > > > >>>> > > > enlight
> > > > >>>> > > > > > >> future work, for that
package, for Clojure
> > developers.
> > > > >>>> > > > > > >>
> > > > >>>> > > > > > >>> On Mon, Jun 4,
2018 at 8:50 PM, Naveen Swamy <
> > > > >>>> > mnnaveen@gmail.com
> > > > >>>> > > >
> > > > >>>> > > > > > wrote:
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>> Hi Carin,
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>> This is great effort
and very nicely documented.
> > When
> > > I
> > > > >>>> > surveyed
> > > > >>>> > > at
> > > > >>>> > > > > my
> > > > >>>> > > > > > >> day
> > > > >>>> > > > > > >>> job on the interest
for Scala packages there were
> a
> > > few
> > > > >>>> people
> > > > >>>> > > who
> > > > >>>> > > > > > >>> specifically were
interested in the Clojure
> > packages,
> > > I
> > > > >>>> think
> > > > >>>> > > those
> > > > >>>> > > > > are
> > > > >>>> > > > > > >>> people who might
have tried and understand the
> > > > complexity
> > > > >>>> of
> > > > >>>> > > making
> > > > >>>> > > > > > Scala
> > > > >>>> > > > > > >>> work in Clojure.
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>> I am in support
of a MXNet-Clojure package, I
> > request
> > > > you
> > > > >>>> to
> > > > >>>> > add
> > > > >>>> > > > more
> > > > >>>> > > > > > >> unit
> > > > >>>> > > > > > >>> tests and integration
tests that can be ported to
> > CI.
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>> Thanks for your
efforts.
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>> -Naveen
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>> On Fri, Jun 1,
2018 at 6:16 PM, Carin Meier <
> > > > >>>> > > carinmeier@gmail.com>
> > > > >>>> > > > > > >> wrote:
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>>> It is always
a good thing to consider the cost
> with
> > > the
> > > > >>>> > benefit.
> > > > >>>> > > > > I'll
> > > > >>>> > > > > > >> do
> > > > >>>> > > > > > >>> my
> > > > >>>> > > > > > >>>> best to explain
what I see the tradeoffs to be.
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>> First, I wanted
to clarify that it took
> significant
> > > > >>>> > development
> > > > >>>> > > > > effort
> > > > >>>> > > > > > >> to
> > > > >>>> > > > > > >>>> get the Clojure
package and the interop working
> > > > properly
> > > > >>>> > despite
> > > > >>>> > > > my
> > > > >>>> > > > > > >>> simple
> > > > >>>> > > > > > >>>> looking design
on the confluence page :)
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>> One of the
advantages of MXNet over its
> competitors
> > > is
> > > > its
> > > > >>>> > many
> > > > >>>> > > > > > >> language
> > > > >>>> > > > > > >>>> support. The
Clojure package would only increase
> > the
> > > > >>>> value of
> > > > >>>> > > this
> > > > >>>> > > > > > >>>> proposition
and bring new users and growth into
> the
> > > > >>>> community.
> > > > >>>> > > > > > >>>> However, there
is a cost associated with adding
> > this
> > > > >>>> language
> > > > >>>> > > > > support
> > > > >>>> > > > > > >> as
> > > > >>>> > > > > > >>>> you pointed
out.
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>> Since the Clojure
package right now is only
> reliant
> > > on
> > > > the
> > > > >>>> > Scala
> > > > >>>> > > > > jars
> > > > >>>> > > > > > >>> from
> > > > >>>> > > > > > >>>> Maven, it can
exist outside the main project as
> an
> > > > >>>> independent
> > > > >>>> > > > repo
> > > > >>>> > > > > > >> but I
> > > > >>>> > > > > > >>>> think that
would lessen the growth benefit both
> to
> > > the
> > > > >>>> Clojure
> > > > >>>> > > > > > >> community
> > > > >>>> > > > > > >>>> and to the
MXNet community to not be included as
> a
> > > > first
> > > > >>>> class
> > > > >>>> > > > > > >> language.
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>> I believe having
first class Clojure support in
> > MXNet
> > > > is
> > > > >>>> > > valuable,
> > > > >>>> > > > > but
> > > > >>>> > > > > > >>> the
> > > > >>>> > > > > > >>>> cost of that
support is up to the community to
> > > decide.
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>> Is there a
process for considering a new package
> in
> > > > MXNet?
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>> - Carin
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>>> On Fri,
Jun 1, 2018 at 5:51 PM, Chen HY <
> > > > >>>> > chenhy12345@gmail.com
> > > > >>>> > > >
> > > > >>>> > > > > > wrote:
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>> Have checked
the issue and the confluence page,
> > but
> > > > still
> > > > >>>> > > > curious.
> > > > >>>> > > > > > >>>>> Clojure
and Scala are both JVM based languages.
> > > > >>>> > > > > > >>>>> They, as
well as many JVM based languages, can
> > share
> > > > >>>> their
> > > > >>>> > > class
> > > > >>>> > > > > and
> > > > >>>> > > > > > >>>> method
> > > > >>>> > > > > > >>>>> at a certain
level.
> > > > >>>> > > > > > >>>>> Why should
the community maintain two APIs for
> two
> > > > >>>> languages
> > > > >>>> > > with
> > > > >>>> > > > > can
> > > > >>>> > > > > > >>>> share
> > > > >>>> > > > > > >>>>> their packages
with almost zero effort?
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>> 2018-06-01
21:58 GMT+01:00 Carin Meier <
> > > > >>>> carinmeier@gmail.com
> > > > >>>> > >:
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>>> Hi
all,
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> I've
been working on a Clojure package for
> MXNet.
> > > > Since
> > > > >>>> > > Clojure
> > > > >>>> > > > is
> > > > >>>> > > > > > >> a
> > > > >>>> > > > > > >>>> JVM
> > > > >>>> > > > > > >>>>>> language,
the package leverages the great work
> of
> > > the
> > > > >>>> > existing
> > > > >>>> > > > > > >> Scala
> > > > >>>> > > > > > >>>>>> package.
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> I would
appreciate any feedback and testing.
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> Here
is the original issue:
> > > > >>>> > > > > > >>>>>> https://github.com/apache/
> > > > incubator-mxnet/issues/8971
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> Architecture
& Design:
> > > > >>>> > > > > > >>>>>> https://cwiki.apache.org/
> > confluence/display/MXNET/
> > > > >>>> > > MXNet+Clojure
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> and
the github repo for rapid testing and issue
> > > > fixing
> > > > >>>> > before
> > > > >>>> > > of
> > > > >>>> > > > > > >>>> opening
> > > > >>>> > > > > > >>>>> an
> > > > >>>> > > > > > >>>>>> official
PR https://github.com/gigasquid/
> > > > clojure-mxnet
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> I'm
also active in the slack channel so feel
> free
> > > to
> > > > >>>> ping me
> > > > >>>> > > > > there.
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>> Thanks,
> > > > >>>> > > > > > >>>>>> Carin
Meier
> > > > >>>> > > > > > >>>>>>
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>> --
> > > > >>>> > > > > > >>>>> Chen Hanyang
陈涵洋
> > > > >>>> > > > > > >>>>> Software
School Fudan University
> > > > >>>> > > > > > >>>>> +86-138-1881-7745
> > > > >>>> > > > > > >>>>>
> > > > >>>> > > > > > >>>>
> > > > >>>> > > > > > >>>
> > > > >>>> > > > > > >>
> > > > >>>> > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > > > >
> > > > >>>> > > > > > > --
> > > > >>>> > > > > > > Sandeep Krishnamurthy
> > > > >>>> > > > > >
> > > > >>>> > > > >
> > > > >>>> > > >
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> --
> > > > >>>> Chen Hanyang 陈涵洋
> > > > >>>> Software School Fudan University
> > > > >>>> +86-138-1881-7745
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Yizhi Liu
> > > > DMLC member
> > > > Amazon Web Services
> > > > Vancouver, Canada
> > > >
> > >
> >
> >
> >
> > --
> > Chen Hanyang 陈涵洋
> > Software School Fudan University
> > +86-138-1881-7745
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message