kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a...@apache.org
Subject [kudu] branch master updated: build-and-test.sh: fix Python 2.7 build
Date Wed, 11 Dec 2019 01:22:01 GMT
This is an automated email from the ASF dual-hosted git repository.

adar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/master by this push:
     new 6ac9191  build-and-test.sh: fix Python 2.7 build
6ac9191 is described below

commit 6ac9191c731d6955a97f08d4add02a96198f6fbf
Author: Adar Dembo <adar@cloudera.com>
AuthorDate: Tue Dec 10 13:57:49 2019 -0800

    build-and-test.sh: fix Python 2.7 build
    
    When building with Python 2.7, the new setuptools 42.0.0 breaks the
    kudu-python build with an error like this:
    
      [1/3] Cythonizing kudu/client.pyx
      [2/3] Cythonizing kudu/errors.pyx
      [3/3] Cythonizing kudu/schema.pyx
      WARNING: The wheel package is not available.
      ERROR: 'pip wheel' requires the 'wheel' package. To fix this, run: pip install wheel
      Traceback (most recent call last):
      File "setup.py", line 216, in <module>
        test_suite="kudu.tests"
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/__init__.py",
line 144, in setup
        _install_setup_requires(attrs)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/__init__.py",
line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/dist.py",
line 721, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 782, in resolve
        replace_conflicting=replace_conflicting
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 1065, in best_match
        return self.obtain(req, installer)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
line 1077, in obtain
        return installer(requirement)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/dist.py",
line 777, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/local/lib/python2.7/site-packages/setuptools/installer.py",
line 130, in fetch_build_egg
        raise DistutilsError(str(e))
      distutils.errors.DistutilsError: Command '['/home/jenkins-slave/workspace/kudu-master/3/build/debug/py_env/bin/python',
'-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpK8eEW3',
'--quiet', 'pytest-runner']' returned non-zero exit status 1
    
    An easy fix is to pin to an earlier version of setuptools. I don't know why
    this doesn't happen on Python 3.
    
    Change-Id: Ia37e164f0319236265165748de47e414a057338f
    Reviewed-on: http://gerrit.cloudera.org:8080/14875
    Reviewed-by: Andrew Wong <awong@cloudera.com>
    Reviewed-by: Bankim Bhavsar <bankim@cloudera.com>
    Reviewed-by: Alexey Serbin <aserbin@cloudera.com>
    Tested-by: Adar Dembo <adar@cloudera.com>
---
 build-support/jenkins/build-and-test.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/build-support/jenkins/build-and-test.sh b/build-support/jenkins/build-and-test.sh
index 11f6336..8844684 100755
--- a/build-support/jenkins/build-and-test.sh
+++ b/build-support/jenkins/build-and-test.sh
@@ -504,13 +504,16 @@ if [ "$BUILD_PYTHON" == "1" ]; then
   #
   # The absence of $PIP_FLAGS is intentional: older versions of pip may not
   # support the flags that we want to use.
-  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip < 10.0.0b1'
+  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip <10.0.0b1'
 
   # New versions of pip raise an exception when upgrading old versions of
   # setuptools (such as the one found in el6). The workaround is to upgrade
   # setuptools on its own, outside of requirements.txt, and with the pip version
   # check disabled.
-  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools
>= 0.8'
+  #
+  # Setuptools 42.0.0 changes something that causes build_ext to fail with a
+  # missing wheel package. Let's pin to an older version known to work.
+  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools
>=0.8,<42.0.0'
 
   # One of our dependencies is pandas, installed below. It depends on numpy, and
   # if we don't install numpy directly, the pandas installation will install the
@@ -521,7 +524,7 @@ if [ "$BUILD_PYTHON" == "1" ]; then
   # so we must pass in the current values of CC and CXX.
   #
   # See https://github.com/numpy/numpy/releases/tag/v1.12.0 for more details.
-  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy < 1.12.0'
+  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy <1.12.0'
 
   # We've got a new pip and new setuptools. We can now install the rest of the
   # Python client's requirements.
@@ -538,7 +541,7 @@ if [ "$BUILD_PYTHON" == "1" ]; then
   #
   # pandas 0.18 dropped support for python 2.6. See https://pandas.pydata.org/pandas-docs/version/0.23.0/whatsnew.html#v0-18-0-march-13-2016
   # for more details.
-  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'pandas < 0.18'
+  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'pandas <0.18'
 
   # Delete old Cython extensions to force them to be rebuilt.
   rm -Rf build kudu_python.egg-info kudu/*.so
@@ -593,13 +596,13 @@ if [ "$BUILD_PYTHON3" == "1" ]; then
   # support the flags that we want to use.
   #
   # 1. https://github.com/pypa/pip/issues/6175
-  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip < 19.0'
+  pip install -i https://pypi.python.org/simple $PIP_INSTALL_FLAGS --upgrade 'pip <19.0'
 
   # New versions of pip raise an exception when upgrading old versions of
   # setuptools (such as the one found in el6). The workaround is to upgrade
   # setuptools on its own, outside of requirements.txt, and with the pip version
   # check disabled.
-  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools
>= 0.8'
+  pip $PIP_FLAGS install --disable-pip-version-check $PIP_INSTALL_FLAGS --upgrade 'setuptools
>=0.8'
 
   # One of our dependencies is pandas, installed below. It depends on numpy, and
   # if we don't install numpy directly, the pandas installation will install the
@@ -611,7 +614,7 @@ if [ "$BUILD_PYTHON3" == "1" ]; then
   # so we must pass in the current values of CC and CXX.
   #
   # See https://github.com/numpy/numpy/releases/tag/v1.16.0rc1 for more details.
-  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy < 1.16.0'
+  CC=$CLANG CXX=$CLANG++ pip $PIP_FLAGS install $PIP_INSTALL_FLAGS 'numpy <1.16.0'
 
   # We've got a new pip and new setuptools. We can now install the rest of the
   # Python client's requirements.


Mime
View raw message