From dev-return-2304-archive-asf-public=cust-asf.ponee.io@mxnet.incubator.apache.org Wed Mar 7 00:13:43 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 63A97180652 for ; Wed, 7 Mar 2018 00:13:42 +0100 (CET) Received: (qmail 38762 invoked by uid 500); 6 Mar 2018 23:13:41 -0000 Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mxnet.incubator.apache.org Delivered-To: mailing list dev@mxnet.incubator.apache.org Received: (qmail 38746 invoked by uid 99); 6 Mar 2018 23:13:40 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Mar 2018 23:13:40 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 3B4C6C0D6E for ; Tue, 6 Mar 2018 23:13:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id MQySsI6q-cHw for ; Tue, 6 Mar 2018 23:13:38 +0000 (UTC) Received: from mail-yw0-f182.google.com (mail-yw0-f182.google.com [209.85.161.182]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id BF60E5F201 for ; Tue, 6 Mar 2018 23:13:37 +0000 (UTC) Received: by mail-yw0-f182.google.com with SMTP id p70so124831ywg.10 for ; Tue, 06 Mar 2018 15:13:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=IQaCEUkZoBRm4zraHHVfxrEiLsT/lwC+0nmUk4u4p9M=; b=CM//KJ67GnRsGHCrMXYauwT8WUJ8dxtsJASuOl7DjVwvwNe9GZJfjQk43ogctbNhfn jxBpJTDzFUZzYF8R0rw5bQt+B1wrrIg9JTHSUb3CaQJe2wDEDZSn6ps7Il8jNgB80DUm 6zc1y7YEXgwtEi0VN4mNgd3a0X3YfXAhtz7VTZLkcEBLQwXe69/6sxho3LG5ar8WR1b9 QABz5qwIhm9P6QM+fTAJZPOEPZTwxu3kefzuR8BP0YIY6pBKGiJt+G6XhV0Yf5K0lZVU 8g1dsmcPf4424KZkm4w8tpSWnKUy6Pjb8sSENdzfV/j8NTyG+vU/EE5nBx/QN42DhrZ6 O+rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=IQaCEUkZoBRm4zraHHVfxrEiLsT/lwC+0nmUk4u4p9M=; b=Sslb5FbUfQ2FNc2QXQZOQhq1pXTWUGY2A8SqD1fMUZUnUJ+lNUuAcwgGXY65A9hrwj kLYey0Am8tsT0I/S8qeoicJYe8yP/5n4juFQKok4XGafXYkGySxj/SYiYQLmRac5Xmb7 KmK5G68ZW9cvGfLhmspT9lZNf3+tsJ/uO90tObWby9hIY0IKQJ5t65S6DDqreUgvAyxw fFbkouHvE5CcjFzXbBKQhWA7SzyjqYeY27hNKLZJ+zylr8yhkFAesluIN/h0f0PkJUAn nPLPKB4j9khmJqmwAND3yZMZJm7LJYhRYDdm7AVELeoJJ527GEVC5bAxFy9SxG1G6oFx mDuw== X-Gm-Message-State: APf1xPAYEFV/8hxmz0r4yZfvSA/FAtSXMc+lo1AfYKE3iKOSUOGeg6vo v/sORpaCkmI3Cpn4MD5KTd/xqkNipEwwfvha9MM= X-Google-Smtp-Source: AG47ELsL7udipoMF7Uc9QDU+qbJ4uA8S9yIxcx50ZNFFPpmDUz+S4sfiixY4e2dUX8vfod0FRpmsSUsi3CVGstQUnLg= X-Received: by 10.13.224.6 with SMTP id j6mr12995203ywe.484.1520378010310; Tue, 06 Mar 2018 15:13:30 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a25:908c:0:0:0:0:0 with HTTP; Tue, 6 Mar 2018 15:13:29 -0800 (PST) In-Reply-To: References: From: kellen sunderland Date: Wed, 7 Mar 2018 00:13:29 +0100 Message-ID: Subject: Re: Following semantic versioning To: dev@mxnet.incubator.apache.org Content-Type: multipart/alternative; boundary="94eb2c075be83a9d270566c697bc" --94eb2c075be83a9d270566c697bc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I also don't see any reason it wouldn't work. I wonder if we could do this and then offer a convenient pip package with operators in separated .SOs, each linked against various libs. For example all versions of blas libs, all versions of cuda. We could then detected the user's environment at runtime and load the appropriate lib (say MKL cpu ops, or CUDA 9.1 w/ CUDNN7). On Tue, Mar 6, 2018 at 11:45 PM, Chris Olivier wrote: > The static init of your module should register your operator just as it > does for the operators in mxnet (NNVM_REGISTER_OP). While I haven't done > it personally, I see no reason why it wouldn't work like any other operat= or > at that point. > > On Tue, Mar 6, 2018 at 1:28 PM, Barber, Christopher < > Christopher.Barber@analog.com> wrote: > > > There are two separate issues: how to compile against the MXNet source > and > > how to dynamically load external dynamic libraries. While it would be > nice > > to have all necessary headers in the same place, it doesn't really matt= er > > that much if people building extensions have to have access to the enti= re > > MXNet source tree. The real issue is whether you support the ability to > > dynamically load such extensions. > > > > - Christopher > > > > =EF=BB=BFOn 3/6/18, 4:12 PM, "Chris Olivier" wr= ote: > > > > This was discussed in the past and so far, it seems possible for Un= ix > > builds, although it=E2=80=99s going to be messy since the compile w= ould > > generally > > need access to all a large portion of the headers in the source tre= e, > > since > > the =E2=80=9Cthings needed to create your own op=E2=80=9D aren=E2= =80=99t necessarily all > > contained > > in the include/mxnet directory. > > > > On Tue, Mar 6, 2018 at 11:40 AM kellen sunderland < > > kellen.sunderland@gmail.com> wrote: > > > > > Could we actually just define a mechanism so the libs could > register > > their > > > ops at runtime? No linking required? > > > > > > On Tue, Mar 6, 2018, 8:36 PM Pedro Larroy < > > pedro.larroy.lists@gmail.com> > > > wrote: > > > > > > > This is a good point. What additional blockers would there be f= or > > linking > > > > against a user provided library with custom operators? > > > > > > > > > > > > > > > > On Tue, Mar 6, 2018 at 5:16 PM, Barber, Christopher < > > > > Christopher.Barber@analog.com> wrote: > > > > > > > > > To avoid this kind of problem, you really need to support > > features that > > > > > allow MXNet to be extended without having to resort to forkin= g. > > There > > > is > > > > > currently no way to add C++ custom operators without forking, > > and no > > > way > > > > to > > > > > share such operators across projects. This creates a perverse > > incentive > > > > to > > > > > try to get changes that may not belong into the main product. > > > > > > > > > > On 3/5/18, 6:26 PM, "Marco de Abreu" < > > marco.g.abreu@googlemail.com> > > > > > wrote: > > > > > > > > > > Hello, > > > > > > > > > > we recently had a few cases in which it has been attempte= d > > to add > > > new > > > > > functionality to old branches because a customer of > somebodys > > > > $DAY_JOB > > > > > requested it and was unable to switch to the latest relea= se > > or that > > > > > certain > > > > > feature did not make it into the release. This lead to > quite > > a lot > > > of > > > > > discussions and there was no clear standing within the > > community. > > > > > > > > > > Just to cite semantic versioning: > > > > > > > > > > 1. MAJOR version when you make incompatible API change= s, > > > > > 2. MINOR version when you add functionality in a > > > > > backwards-compatible > > > > > manner, and > > > > > 3. PATCH version when you make backwards-compatible bu= g > > fixes. > > > > > > > > > > > > > > > We as a community agreed on following this system and I > > think we > > > > should > > > > > either stick to it or drop it entirely - exceptions to > > SemVer are > > > > > usually > > > > > discouraged. While I see that adding functionality might = be > > a minor > > > > > thing, > > > > > I don't think that we should educate our users into > > expecting us to > > > > > backport new features. The development happening on the > > Apache > > > MXNet > > > > > repository should not be influenced by something like thi= s; > > > > especially > > > > > considering that whoever supports that customer on their > > $DAY_JOB > > > can > > > > > assist them at creating a fork and cherrypicking that > > feature. But > > > I > > > > > don't > > > > > see much reason why we're running against best pracitices= . > > One > > > > > important > > > > > thing to note here is that we're not maintaining CI on ol= d > > branches > > > > and > > > > > neither are we making patch releases - so what do these > > users do? > > > > > Compile > > > > > off a stale development branch with unvalidated changes? > > > > > > > > > > In my opinion this whole topic is just causing trouble an= d > > > > > fragementation > > > > > on our end. If a features doesn't make it into the releas= e > > (for > > > > > whatever > > > > > reason), so be it. But I think we should discuss this top= ic > > and > > > > > emphasize a > > > > > no-exceptions-rule to SemVer. > > > > > > > > > > Best regards, > > > > > Marco > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --94eb2c075be83a9d270566c697bc--