yetus-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: [DISCUSS] Requiring Python 2.7
Date Tue, 06 Sep 2016 17:23:00 GMT
I’m just catching up on this thread now.

I fall in the camp that would prefer not to dictate a requirement of Python 2.7 for anything
in Yetus.  My reasoning agrees with what Allen has already described.  I have been in the
situation that he describes, needing to maintain build infrastructure on old OSes and old
Python versions.  Upgrading the OS or installing Python 2.7 via conda or a custom rpm was
not an option.  IT policies prevented me from performing non-essential/non-security related
upgrades or installing new software.  Instead, the solution was to stick to Python code compatible
with the old versions.

I haven’t reviewed the code for YETUS-445, but the motivation for this topic coming up now
seems to be argparse.  I think Ajay’s proposal for handling that sounds fine.  Alternatively,
we could explore if it’s feasible to check in a copy of argparse to be front-loaded onto
PYTHONPATH.

--Chris Nauroth

On 8/24/16, 9:45 PM, "Ajay Yadava" <ajayyadavaatos@gmail.com> wrote:

    If the only motivation for Python 2.7 for this script is to use argparse
    library then it can be installed on Python 2.6 and hence shouldn't be an
    issue IMO. We have external dependencies for RDM as well. A simple
    try-catch will ensure that 2.6 users know about the external dependency.
    Does this work?
    
    On Thu, Aug 25, 2016, 9:22 AM Dima Spivak <dspivak@cloudera.com> wrote:
    
    > What if we compromise by creating some sort of designation for tools
    > that require runs to use Docker mode? This way, people could still use
    > Andrew's awesome script either on their own or in Apache Infra (even
    > precommit!) without unfortunate surprises when someone tries to run it on
    > an older OS.
    >
    > On Wednesday, August 24, 2016, Andrew Wang <andrew.wang@cloudera.com>
    > wrote:
    >
    > > On Wed, Aug 24, 2016 at 4:46 PM, Allen Wittenauer <
    > > aw@effectivemachines.com <javascript:;>>
    > > wrote:
    > >
    > > >
    > > > > On Aug 24, 2016, at 3:49 PM, Andrew Wang <andrew.wang@cloudera.com
    > > <javascript:;>>
    > > > wrote:
    > > > > My reasoning here has been that this tool will primarily be run by
    > > > release
    > > > > managers, and RMs are very likely to have Python 2.7 on their
    > machine.
    > > >
    > > >         Whereas I believe the opposite: there's a good chance if one is
    > > > the RM for enterprise software, the machine you are building on
    > probably
    > > > doesn't have python 2.7 on it because you aren't doing this work on
    > your
    > > > desktop!  It's extremely common to have build servers that are several
    > OS
    > > > revs behind because software tends to be more upwardly compatible....
    > > >
    > > > > ASF infra and servers in general are not really a target.
    > > >
    > > >         Umm, yeah, they kind of are. While there are certainly issues
    > > with
    > > > them, they do serve as sort of a model of what is out in the real
    > world.
    > > > It's old and crufty and that's how a lot of software is built.  Where
    > > > people build software is *exactly* our target.
    > > >
    > > >         Here's a fun anecdote.  Firefox v48 is the first version to
    > drop
    > > > support for Mac OS X 10.6, 10.7, and 10.8.  This means they were almost
    > > > certainly building on 10.6.  So up until earlier this year, they were
    > > using
    > > > a 7 year old OS that most definitely didn't have python 2.7 on it by
    > > > default. Mozilla, by many accounts, would be considered quite
    > aggressive
    > > > but they can get away with it because they are targeting the desktop.
    > > >
    > > >
    > > I think we're talking past each other here, since this is not a build
    > tool.
    > > It's an API checking tool. It only needs to be run on a developer's
    > > computer, not all boxes at ASF. And, like I've been saying, you can just
    > > install Python 2.7 like you would any other dependency. RDM requires `pip
    > > install python-dateutils` for instance.
    > >
    > >
    > > > > Maybe one day if
    > > > > we want to run this as part of precommit, but then it can be optional
    > > > like
    > > > > gradle and docker.
    > > >
    > > >
    > > >         It sends really bizarre mixed messages if one tool has
    > different
    > > > python requirements than another just because someone wanted to use a
    > > > different option parsing library.
    > >
    > >
    > > I don't know about "really bizzare"; this is the same problem as any
    > > external dependency (e.g. python-dateutils).
    > >
    > > The real issue here is the lack of a standardized runtime environment. Is
    > > there build/packaging for Yetus? Some of this could be handled by
    > wrappers
    > > that created virtualenvs or downloaded conda as appropriate.
    > >
    >
    >
    > --
    > -Dima
    >
    -- 
    Regards
    Ajay Yadava
    

Mime
View raw message