aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Smith <yasumo...@gmail.com>
Subject Re: Should we continue to maintain fine-grained python BUILD targets?
Date Wed, 07 Jan 2015 19:27:11 GMT

> On Jan 7, 2015, at 11:24 AM, Kevin Sweeney <ksweeney@twitter.com.INVALID> wrote:
> 
> +1 to Josh's statement - if we're going to stay with pants we should follow
> the 1 directory, 1 package, 1 BUILD file convention (though if we stay the
> course there I suggest we write a tool to generate BUILD files based on
> static analysis of import statements).
> 
> +1 to exploring tox+setup.py and replacing pants completely (or at least
> for everything but thrift codegen).
> 
> To Joe's point: we can still select individual tests with the -k flag to
> py.test/tox (that's pretty much what happens when you right-click > run
> test in pycharm), so we don't lose the ability to run tests granularly in
> either scenario.

Aha, good point, thanks for the hint :)


> 
> On Wed, Jan 7, 2015 at 11:09 AM, John Sirois <john.sirois@gmail.com> wrote:
> 
>> On Wed, Jan 7, 2015 at 12:05 PM, Joseph Smith <yasumoto7@gmail.com> wrote:
>> 
>>> I found it ~annoying to test the client in a few cases where multiple
>>> targets were in one large test target- I’m in favor of fine-grained BUILD
>>> targets despite the (what I consider slight) additional overhead.
>>> 
>> 
>> NB though that you can use pytest's -k to select individual tests no matter
>> how many files or targets are passed under tox and pants respectively, so I
>> think this particular argument for or against washes out.
>> 
>> 
>>>> On Jan 7, 2015, at 10:14 AM, Joshua Cohen <jcohen@twopensource.com>
>>> wrote:
>>>> 
>>>> I agree, pants is heavily biased towards 1 directory, 1 package, 1
>> build
>>>> target. If we're going to stay with pants I'd prefer to stick with that
>>>> convention. If we're going to simplify the build structure I think we
>>>> should take that as impetus to move away from pants entirely.
>>>> 
>>>> On Tue, Jan 6, 2015 at 2:43 PM, Brian Wickman <wickman@apache.org>
>>> wrote:
>>>> 
>>>>> 4) Enforcing sensible software hygiene.
>>>>> 
>>>>> The pants project itself for the longest time had a single root rglobs
>>>>> python_library target.  I spent a few days splitting up that project
>>> into
>>>>> fine-grained targets and it uncovered a number of cyclical
>> dependencies
>>> and
>>>>> abstraction leaks that simply would've been impossible to make with
>>> better
>>>>> target encapsulation.  If everyone is an expert in the project, BUILD
>>>>> management can be tedious overhead, but for novice contributors,
>>>>> fine-grained BUILDs can be an effective linting mechanism to prevent
>>> poor
>>>>> code structure and advise the design process.
>>>>> 
>>>>> This is entirely a trade-off.  I'm not sure you could get the same
>>>>> protections with coarser granularity than what we have now.  That
>> being
>>>>> said, if we move forward with simplification, I'd propose we go all-in
>>> and
>>>>> remove pants entirely in favor of setup.py and tox.
>>>>> 
>>>>> ~brian
>>>>> 
>>>>> 
>>>>> 
>>>>> On Tue, Jan 6, 2015 at 2:03 PM, Bill Farner <wfarner@apache.org>
>> wrote:
>>>>> 
>>>>>> We currently have ~111 BUILD targets serving our python code, and
>> ~106
>>>>> test
>>>>>> targets:
>>>>>> 
>>>>>> $ find . -name BUILD | xargs grep python_library | wc -l
>>>>>> 111
>>>>>> 
>>>>>> $ find . -name BUILD | xargs grep python_test | wc -l
>>>>>> 106
>>>>>> 
>>>>>> This presents a non-trivial maintenance burden, and i'm not convinced
>>> the
>>>>>> benefits are worth it.  Now that we have a good IDE story, refactors
>>> are
>>>>>> painless in the source code, but painful when you need to hand-edit
>> all
>>>>>> affected BUILD targets.
>>>>>> 
>>>>>> I can think of several motivations for using fine-grained targets
>>>>>> 1. Running specific unit tests with pants
>>>>>> 2. Limiting the sources included in a binary
>>>>>> 3. Carving out libraries to be published
>>>>>> 
>>>>>> I'm not convinced these benefits outweigh the maintenance burden
of
>>> these
>>>>>> files.  I propose we reduce our BUILD targets to ~equal to the number
>>> of
>>>>>> binaries we produce, possibly with a few for libraries shared between
>>>>>> components.
>>>>>> 
>>>>>> Does anyone else have opinions on this?  Is there something i'm
>>>>>> overlooking?
>>>>>> 
>>>>>> 
>>>>>> -=Bill
>>>>>> 
>>>>> 
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Kevin Sweeney
> @kts


Mime
View raw message