impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Knupp (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5181: Extract PYPI metadata from a webpage
Date Fri, 07 Apr 2017 18:40:06 GMT
David Knupp has posted comments on this change.

Change subject: IMPALA-5181: Extract PYPI metadata from a webpage
......................................................................


Patch Set 1:

There may well be some historical context that I'm missing here, or just some
complication that I'm overlooking, so sincere apologies if these comments are
not relevant, but one thing I've noticed is that even though we create a python
virtualenv, instead of ever activating it, we use wrapper scripts to construct
elaborate paths to the enclosed binaries (not just Impala does this, but other
Cloudera teams do this too.)

E.g., if I'm in my system evironment:

  dknupp@dknupp-desktop:0:~ $ which python pip virtualenv
  /usr/bin/python
  /usr/local/bin/pip
  /usr/local/bin/virtualenv
  dknupp@dknupp-desktop:0:~ $ pip --version
  pip 8.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)
  dknupp@dknupp-desktop:0:~ $ virtualenv --version
  15.0.2  

But if I _activate_ the Impala python virtualenv first...

  dknupp@dknupp-desktop:0:~ $ source ${IMPALA_HOME}/infra/python/env/bin/activate
  (env)dknupp@dknupp-desktop:0:~ $ which python pip virtualenv
  /home/dknupp/Impala/infra/python/env/bin/python
  /home/dknupp/Impala/infra/python/env/bin/pip
  /home/dknupp/Impala/infra/python/env/bin/virtualenv

...the system versions no longer matter, and I can upgrade these at will.

  (env)dknupp@dknupp-desktop:0:~ $ pip install -U pip virtualenv
  Collecting pip
    Using cached pip-9.0.1-py2.py3-none-any.whl
  Collecting virtualenv
    Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
      100% |################################| 1.8MB 1.0MB/s
  Installing collected packages: pip, virtualenv
    Found existing installation: pip 8.1.2
      Uninstalling pip-8.1.2:
        Successfully uninstalled pip-8.1.2
    Found existing installation: virtualenv 13.1.0
      Uninstalling virtualenv-13.1.0:
        Successfully uninstalled virtualenv-13.1.0
  Successfully installed pip-9.0.1 virtualenv-15.1.0

  (env)dknupp@dknupp-desktop:0:~ $ pip --version
  pip 9.0.1 from /home/dknupp/Impala/infra/python/env/local/lib/python2.7/site-packages (python
2.7)
  (env)dknupp@dknupp-desktop:0:~ $ virtualenv --version
  15.1.0

When I deactivate the virtualenv, the system versions are untouched.

  dknupp@dknupp-desktop:0:~ $ pip --version
  pip 8.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)
  dknupp@dknupp-desktop:0:~ $ virtualenv --version
  15.0.2

So a possible workflow would be to:

1. Create the Impala virtualenv in ${IMPALA_HOME}/infra/python/env first
2. Activate the virtualenv
3. "pip install -U pip" to upgrade ${IMPALA_HOME}/infra/python/env/bin/pip
4. Using the upgraded pip to download & install the various things
5. Deactivate the virtualenv

-- 
To view, visit http://gerrit.cloudera.org:8080/6579
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: If3845a0d5f568d4352e3cc4883596736974fd7de
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: David Knupp <dknupp@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovytsky@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: No

Mime
View raw message