mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Huilgol <rahulhuil...@gmail.com>
Subject Re: CI system seems to be using python3 for python2 builds
Date Thu, 28 Sep 2017 06:34:35 GMT
Hi Gautam,

I see that ‘nosetests’ is the command used to run python2 tests. It looks
like that’s being mapped to use python3. I’ve checked that this is the case
on my Ubuntu instance. I need to use ‘nosetests-2.7’ to use python2 for the
tests. Please check if this fix works in the build environment
(slave/docker container) as well.

The PR you refer to only parallelized tests that were running one after the
other, this command was being used even before that PR.

Regards,
Rahul

On Wed, 27 Sep 2017 at 22:46 Gautam <gautamnitc@gmail.com> wrote:

> Hi Ozawa,
>
>   Thanks for follow up.
>   Unfortunately I didn't get time to work on this today.
>
> However I have couple of points to mentions.
> 1. Looks like this backtrace has been present since long time, since this
> was not a test failure or build failure we never got notified about it.
> Here
> <
> https://builds.apache.org/view/Incubator%20Projects/job/incubator-mxnet/job/master/448/consoleFull
> >
> is the recent build log where back trace is present but build succeeded.
>
> 2. I don't think the default version of python on Ubuntu is 3.0, I logged
> into one of the apache slave and the default version of Python is 2.7.6
>
> 3. There has been slight change
> <https://github.com/apache/incubator-mxnet/pull/7658> in Jenkins file
> where
> we tried to parallelize python2 and 3 test run. I am not sure if it
> affects. I can probably scrub the build log and figure out if thats the
> case.
>
>
> Feel free to send the PR, if you have it ready.
>
>
> -Gautam
>
>
> On Wed, Sep 27, 2017 at 9:39 PM, Tsuyoshi Ozawa <ozawa@apache.org> wrote:
>
> > Hi Kumar,
> >
> > Thanks for looking into the issue. How is the progress of this problem?
> > Shouldn't we call /usr/bin/env python2 or python2.7 in following
> > source code instead of python since MXNet only supports python2
> > currently?
> > I think default version of python in Ubuntu is now python3, so it can
> > cause the problem.
> > If you have not yet done the work, I can create a PR for that in this
> > weekend.
> >
> > ./python/mxnet/__init__.py:#!/usr/bin/env python
> > ./python/mxnet/log.py:#!/usr/bin/env python
> > ./tests/nightly/dist_lenet.py:#!/usr/bin/env python
> > ./tests/nightly/dist_sync_kvstore.py:#!/usr/bin/env python
> > ./tests/nightly/multi_lenet.py:#!/usr/bin/env python
> > ./tests/nightly/test_kvstore.py:#!/usr/bin/env python
> > ./tools/coreml/mxnet_coreml_converter.py:#!/usr/bin/env python
> > ./tools/ipynb2md.py:#!/usr/bin/env python
> > ./tools/kill-mxnet.py:#!/usr/bin/env python
> > ./tools/launch.py:#!/usr/bin/env python
> > ./tools/parse_log.py:#!/usr/bin/env python
> >
> > On Wed, Sep 27, 2017 at 5:39 PM, Sunderland, Kellen <kellens@amazon.de>
> > wrote:
> > > Many thanks Gautam.
> > >
> > > On 9/26/17, 8:37 PM, "Kumar, Gautam" <gauta@amazon.com> wrote:
> > >
> > >     Hi Kellen,
> > >
> > >        This issue has been happening since last 3-4 days along with few
> > other test failure.
> > >     I am looking into it.
> > >
> > >     -Gautam
> > >
> > >     On 9/26/17, 7:45 AM, "Sunderland, Kellen" <kellens@amazon.de>
> wrote:
> > >
> > >         I’ve been noticing in a few failed builds that the stack trace
> > indicates we’re actually running python 3.4 in the python 2 tests. I know
> > the CI folks are working hard getting everything setup, is this a known
> > issue for the CI team?
> > >
> > >         For example: https://builds.apache.org/
> > blue/organizations/jenkins/incubator-mxnet/detail/PR-8026/3/pipeline/281
> > >
> > >         Steps Python2: MKLML-CPU
> > >
> > >         StackTrace:
> > >         Stack trace returned 10 entries:
> > >         [bt] (0) /workspace/python/mxnet/../../lib/libmxnet.so(_
> > ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7fadb8999aac]
> > >         [bt] (1) /workspace/python/mxnet/../../lib/libmxnet.so(_
> > ZN5mxnet7kvstore12KVStoreLocal12GroupKVPairsISt4pairIPNS_
> >
> 7NDArrayES4_EZNS1_19GroupKVPairsPullRspERKSt6vectorIiSaIiEERKS7_IS6_SaIS6_
> >
> EEPS9_PS7_ISD_SaISD_EEEUliRKS6_E_EEvSB_RKS7_IT_SaISN_EESG_PS7_ISP_SaISP_EERKT0_+0x56b)
> > [0x7fadba32c01b]
> > >         [bt] (2) /workspace/python/mxnet/../../lib/libmxnet.so(_
> > ZN5mxnet7kvstore12KVStoreLocal17PullRowSparseImplERKSt6vecto
> > rIiSaIiEERKS2_ISt4pairIPNS_7NDArrayES8_ESaISA_EEi+0xa6) [0x7fadba32c856]
> > >         [bt] (3)
> /workspace/python/mxnet/../../lib/libmxnet.so(MXKVStorePullRowSparse+0x245)
> > [0x7fadba18f165]
> > >         [bt] (4)
> /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c)
> > [0x7fadde26cadc]
> > >         [bt] (5) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x1fc)
> > [0x7fadde26c40c]
> > >         [bt] (6) /usr/lib/python3.4/lib-dynload/_ctypes.cpython-34m-
> > x86_64-linux-gnu.so(_ctypes_callproc+0x21d) [0x7fadde47e12d]
> > >         [bt] (7) /usr/lib/p
> <https://maps.google.com/?q=(7)+/usr/lib/p&entry=gmail&source=g>
> ython3.4/lib-dynload/_ctypes.cpython-34m-
> > x86_64-linux-gnu.so(+0xf6a3) [0x7fadde47e6a3]
> > >         [bt] (8) /usr/bin/python3(PyEval_EvalFrameEx+0x41d7) [0x48a487]
> > >         [bt] (9) /usr/bin/python3() [0x48f2df]
> > >
> > >         -Kellen
> > >         Amazon Development Center Germany GmbH
> > >         Berlin - Dresden - Aachen
> > >         main office: Krausenstr. 38, 10117 Berlin
> > >         Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
> > >         Ust-ID: DE289237879
> > >         Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
> > >
> > >
> > >
> > >
> > > Amazon Development Center Germany GmbH
> > > Berlin - Dresden - Aachen
> > > main office: Krausenstr. 38, 10117 Berlin
> > > Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
> > > Ust-ID: DE289237879
> > > Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
> >
>
>
>
> --
> Best Regards,
> Gautam Kumar
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message