aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Sweeney <kswee...@twitter.com.INVALID>
Subject Re: Should we continue to maintain fine-grained python BUILD targets?
Date Wed, 07 Jan 2015 19:24:49 GMT
+1 to Josh's statement - if we're going to stay with pants we should follow
the 1 directory, 1 package, 1 BUILD file convention (though if we stay the
course there I suggest we write a tool to generate BUILD files based on
static analysis of import statements).

+1 to exploring tox+setup.py and replacing pants completely (or at least
for everything but thrift codegen).

To Joe's point: we can still select individual tests with the -k flag to
py.test/tox (that's pretty much what happens when you right-click > run
test in pycharm), so we don't lose the ability to run tests granularly in
either scenario.

On Wed, Jan 7, 2015 at 11:09 AM, John Sirois <john.sirois@gmail.com> wrote:

> On Wed, Jan 7, 2015 at 12:05 PM, Joseph Smith <yasumoto7@gmail.com> wrote:
>
> > I found it ~annoying to test the client in a few cases where multiple
> > targets were in one large test target- I’m in favor of fine-grained BUILD
> > targets despite the (what I consider slight) additional overhead.
> >
>
> NB though that you can use pytest's -k to select individual tests no matter
> how many files or targets are passed under tox and pants respectively, so I
> think this particular argument for or against washes out.
>
>
> > > On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jcohen@twopensource.com>
> > wrote:
> > >
> > > I agree, pants is heavily biased towards 1 directory, 1 package, 1
> build
> > > target. If we're going to stay with pants I'd prefer to stick with that
> > > convention. If we're going to simplify the build structure I think we
> > > should take that as impetus to move away from pants entirely.
> > >
> > > On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wickman@apache.org>
> > wrote:
> > >
> > >> 4) Enforcing sensible software hygiene.
> > >>
> > >> The pants project itself for the longest time had a single root rglobs
> > >> python_library target.  I spent a few days splitting up that project
> > into
> > >> fine-grained targets and it uncovered a number of cyclical
> dependencies
> > and
> > >> abstraction leaks that simply would've been impossible to make with
> > better
> > >> target encapsulation.  If everyone is an expert in the project, BUILD
> > >> management can be tedious overhead, but for novice contributors,
> > >> fine-grained BUILDs can be an effective linting mechanism to prevent
> > poor
> > >> code structure and advise the design process.
> > >>
> > >> This is entirely a trade-off.  I'm not sure you could get the same
> > >> protections with coarser granularity than what we have now.  That
> being
> > >> said, if we move forward with simplification, I'd propose we go all-in
> > and
> > >> remove pants entirely in favor of setup.py and tox.
> > >>
> > >> ~brian
> > >>
> > >>
> > >>
> > >> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wfarner@apache.org>
> wrote:
> > >>
> > >>> We currently have ~111 BUILD targets serving our python code, and
> ~106
> > >> test
> > >>> targets:
> > >>>
> > >>> $ find . -name BUILD | xargs grep python_library | wc -l
> > >>> 111
> > >>>
> > >>> $ find . -name BUILD | xargs grep python_test | wc -l
> > >>> 106
> > >>>
> > >>> This presents a non-trivial maintenance burden, and i'm not convinced
> > the
> > >>> benefits are worth it.  Now that we have a good IDE story, refactors
> > are
> > >>> painless in the source code, but painful when you need to hand-edit
> all
> > >>> affected BUILD targets.
> > >>>
> > >>> I can think of several motivations for using fine-grained targets
> > >>> 1. Running specific unit tests with pants
> > >>> 2. Limiting the sources included in a binary
> > >>> 3. Carving out libraries to be published
> > >>>
> > >>> I'm not convinced these benefits outweigh the maintenance burden of
> > these
> > >>> files.  I propose we reduce our BUILD targets to ~equal to the number
> > of
> > >>> binaries we produce, possibly with a few for libraries shared between
> > >>> components.
> > >>>
> > >>> Does anyone else have opinions on this?  Is there something i'm
> > >>> overlooking?
> > >>>
> > >>>
> > >>> -=Bill
> > >>>
> > >>
> >
> >
>



-- 
Kevin Sweeney
@kts

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message