beam-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Bradshaw <rober...@google.com>
Subject Re: [DISCUSS] Python static type checkers
Date Tue, 14 Jan 2020 02:15:43 GMT
On Mon, Jan 13, 2020 at 6:04 PM Ahmet Altay <altay@google.com> wrote:
>
> > The most important gain would be compatibility with Google internal code.
> I would like to clarify this. This refers to users of Beam who by default are using pytype
as part of the toolchain. Even though they are internal to a one single company and not vocal
on Beam, they still represent a large group of distinct teams using Beam.
>
> On Mon, Jan 13, 2020 at 5:51 PM Robert Bradshaw <robertwb@google.com> wrote:
>>
>> On Mon, Jan 13, 2020 at 5:34 PM Chad Dombrova <chadrik@gmail.com> wrote:
>> >>
>> >> Pytype seems to detect attribute errors that mypy has not, so it acts as
a kind-of linter in this case.
>> >> Examples:
>> >> https://github.com/apache/beam/pull/10528/files#diff-0cb34b4622b0b7d7256d28b1ee1d52fc
>> >> https://github.com/apache/beam/pull/10528/files#diff-7e4ad8c086414399957cdbea711ebd36
>> >> https://github.com/apache/beam/pull/10528/files#diff-d5c3f4f603204c5c5917d89e90dba53d
>> >> (it also makes pytype more strict in a sense)
>> >
>> > Note that mypy is still not fully passing on master so it's unclear from those
diffs exactly how the two tools differ.  Many of the fixes you've made for pytype look familiar
to me from mypy, but my fixes may not be merged yet.  For example, mypy also does not support
@total_ordering, but my fix for that is still pending.
>>
>> As it seems we have  a workaround to ignore pytype for now, it seems
>> to make the most sense to focus on getting mypy working completely
>> before focusing on that. In the long term, we could look into making
>> pytype a post-commit which would at least be a useful signal for those
>> caring about it, and only make it a precommit if the diff between what
>> it requires and mypy requires is quite small.
>
> The workaround to ignore pytype is making the experience for type users worse than before.
Previously, they could have type checked their pipelines with pytype and pytype would have
attempted to fill in with best effort type information. Now, pytype is instructed to not provide
this information.

Thanks for the clarification--I had assumed that it only suppressed
warnings from this codebase, not that it would disable providing what
it could. (Which, of course, is still less if it can't understand the
code.)

> I agree with focusing one mypy for now, but I would propose soon after, or in parallel
if it will be different folks, to work on pytype and enable it as a first class citizen similar
to mypy. If there will be a large delta between the two then we can decide on what to do next.

If there is a large delta, I wonder if what is needed to provide
sufficient typing on the public API of beam (needed for users) could
be a much smaller subset than that required fully
documenting/typechecking the internals (which is also likely to be
where more of the tricky bits are).

Mime
View raw message