airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ash Berlin-Taylor <...@apache.org>
Subject Re: Proposal: Automatically mark stale PRs in github
Date Mon, 01 Jul 2019 09:55:05 GMT
An example of why I'm not a _huge_ van of stale bot, at least not for issues.

https://github.com/dpgaspar/Flask-AppBuilder/issues/685

That is still an issue but was closed just because no one responded to it.

> On 11 Jun 2019, at 06:50, Jarek Potiuk <Jarek.Potiuk@polidea.com> wrote:
> 
> This issue bugs me a lot. Pretty much all our PRs were updated 2 days ago
> again :(
> 
> I've opened the ticket to Apache Infrastructure
> https://issues.apache.org/jira/browse/INFRA-18589 and I hope we can get to
> the bottom of it. I believe it might be some integration we have (but I
> have no access to it). I looked at other Apache repositories and they do
> not have similar "updates" happening, so it must be something specific for
> apache/airflow repo.
> 
> J.
> 
> On Thu, May 30, 2019 at 10:41 PM Jarek Potiuk <Jarek.Potiuk@polidea.com>
> wrote:
> 
>> Well. Github support is quite far from being helpful :(. We'll have to dig
>> deeper on our own it seems
>> 
>> Our apologies for the wait, and thank you for getting in touch! Due to a
>> high volume of requests, we are currently experiencing much longer than
>> average response times here in Support. You asked:
>> 
>> Can you please let us know what action caused the update and what can we
>> do to prevent it from happening again ?
>> 
>> The updated_at for any object, including users, will change whenever the
>> database record for that object is updated. Such database updates can
>> happen for many reasons, though we don't have a complete list of those to
>> share with you and your team. We wish could be of more help here as we see
>> how this can be a problem for you and your team, but we don't currently
>> have any other insight to share at this time.
>> 
>> Please let us know how else we can be of help!
>> 
>> On Sun, May 19, 2019 at 1:14 PM Jarek Potiuk <Jarek.Potiuk@polidea.com>
>> wrote:
>> 
>>> All our PRs were updated again on Wednesday, 15th of May. I am following
>>> up with Github support (they have not responded so far).
>>> 
>>> Maybe someone happens to know what could have caused the update (some
>>> automated job? bot? CI?). There is absolutely no update visible in the UI
>>> of github for those. I also looked at the fork in some cases - nothing
>>> changed for those either.
>>> 
>>> Or maybe someone has contact at Github so that they verify/fix it faster
>>> ? They must be able to see from the logs what happened to those PRs - from
>>> our point of view looks like most of those PRs were not touched for several
>>> months.
>>> I responded to them with this (the ticket number is 159141).
>>> 
>>> 
>>> Hello GitHub support,
>>> 
>>> We continue to have the same problem. Pretty much all our PR were updated
>>> again 4 days ago - which prevents stalebot from closing them.
>>> 
>>> Example here: https://github.com/apache/airflow/pull/4635  - this PR was
>>> last touched 3 months ago, yet when we list it with this query https://
>>> github
>>> .com/apache/airflow/pulls?page=5&q=is%3Apr+is%3Aopen+updated%3A%3C2019-05-16+sort%3Aupdated-desc&utf8=%E2%9C%93
it
>>> shows as updated 4 days ago (i.e. on Wed 15th of May). I cannot find any
>>> indicatio of a change that could have caused the update date to be bumped
>>> again.
>>> 
>>> Can you please let us know what action caused the update and what can we
>>> do to prevent it from happening again ?
>>> 
>>> J.
>>> 
>>> 
>>> On Mon, May 6, 2019 at 3:54 PM Jarek Potiuk <Jarek.Potiuk@polidea.com>
>>> wrote:
>>> 
>>>> I raised an issue with Github. Let's see what they say:
>>>> 
>>>> Jarek,
>>>> 
>>>> Thank you for contacting GitHub Developer Support. We wanted to let you
>>>> know that we've received your message and will get to it as quickly as
>>>> possible.
>>>> 
>>>> Ticket ID: 159141
>>>> 
>>>> We've also included a copy of your message below.
>>>> 
>>>> If you have any additional information or would like to add anything to
>>>> your initial message, now would be a great time to do so, feel free to
>>>> reply to this email. If not, then rest assured your request is in the right
>>>> hands :)
>>>> 
>>>> Thank you!
>>>> The GitHub Developer Support Team
>>>> 
>>>> *Jarek Potiuk*
>>>> 
>>>> May 6, 1:47 PM UTC
>>>> 
>>>> Hello All,
>>>> 
>>>> In Apache Airflow project we are trying to use stalebot to closed
>>>> not-updated pull requests. And for some reason the bot does not really
>>>> closed our old tickets. We checked what could be wrong and it seems that
>>>> pretty much all our PRs get somehow updated regularly.
>>>> 
>>>> Last time I checked more than 100 PRs were updated at 27th of April and
>>>> yesterday I checked that 118 requests were updated on 28th of April. It
>>>> does not seem that there was any action that could have caused the updates.
>>>> 
>>>> Here are all the requests (all of them updated 27th of April):
>>>> 
>>>> 
>>>> https://github.com/apache/airflow/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+updated%3A%3C2019-04-28+sort%3Aupdated-desc+
>>>> 
>>>> And here is an example PR that was updated 27th of April but there seem
>>>> to be no action that could have caused it:
>>>> https://github.com/apache/airflow/pull/4929
>>>> 
>>>> Can you please explain where the updates are coming from and how we can
>>>> avoid the updates from happening?
>>>> 
>>>> On Mon, May 6, 2019 at 3:39 AM Jiajie Zhong <zhongjiajie955@hotmail.com>
>>>> wrote:
>>>> 
>>>>> It's really odd. I don't know this issue. I think maybe travis-c update
>>>>> our PR time at first but it don't.
>>>>> 
>>>>> BTW, I take a look on some PR and give some example.
>>>>> 
>>>>> https://github.com/apache/airflow/pull/5135 create 17 days ago, last
>>>>> comment 16 days ago, and travis-ci finish 17 days ago (which mean that
CI
>>>>> process don't touch it and change PR update time)
>>>>> 
>>>>> https://github.com/apache/airflow/pull/5136
>>>>> 
>>>>> 
>>>>> Best wish.
>>>>> -- Jiajie
>>>>> ________________________________
>>>>> From: Jarek Potiuk <Jarek.Potiuk@polidea.com>
>>>>> Sent: Monday, May 6, 2019 4:04
>>>>> To: dev@airflow.apache.org
>>>>> Cc: airflowuser
>>>>> Subject: Re: Proposal: Automatically mark stale PRs in github
>>>>> 
>>>>> I believe our current stale bot configuration does not work. And I do
>>>>> not
>>>>> know the reason yet, which worries me :(
>>>>> 
>>>>> There is something really strange going on with our PRs and their
>>>>> updated
>>>>> date. Again pretty much all the PRs were mysteriously updated on *27th
>>>>> of
>>>>> April - 8 days ago* (similarly as the previous case where I saw all PRs
>>>>> updated on *6th of April*).
>>>>> 
>>>>> You can see it here:
>>>>> 
>>>>> * there are just 2(!) PRs updated before 27th of April:
>>>>> 
>>>>> https://github.com/apache/airflow/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+updated%3A%3C2019-04-27+sort%3Aupdated-desc+
>>>>> * there are 120 (!) PRS updated before 28th of April:
>>>>> 
>>>>> https://github.com/apache/airflow/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+updated%3A%3C2019-04-28+sort%3Aupdated-desc+
>>>>> 
>>>>> There is no indication that most of those impacted issues were at all
>>>>> touched on 27th or 28th of April. If you look at random PRs there, most
>>>>> of
>>>>> them were commented latest at the beginning of April.
>>>>> 
>>>>> Looks like 8 days ago some process has bumped the update date for most
>>>>> of
>>>>> our PRs. With this kind of "regular" (it seems) process of marking the
>>>>> requests "updated" our stale bot is useless.
>>>>> 
>>>>> Does anyone have an idea why it might have happened?
>>>>> 
>>>>> I am quite puzzled by this one. I am going to open an issue to Github
>>>>> support if no one has an idea what's going on.
>>>>> 
>>>>> J.
>>>>> 
>>>>> 
>>>>> 
>>>>> On Tue, Apr 23, 2019 at 12:39 PM Jiajie Zhong <
>>>>> zhongjiajie955@hotmail.com>
>>>>> wrote:
>>>>> 
>>>>>> I think we should change stale-bot strategy to auto close PR, If
30
>>>>> days
>>>>>> is too short for contributions, is 60 or 90 days make sence?
>>>>>> 
>>>>>> In addition, I notice that we have some PR pass CI but none review
it
>>>>> or
>>>>>> let a suggest on it. So could we add a bot auto remind committer
if
>>>>> PR pass
>>>>>> CI but no one review?
>>>>>> 
>>>>>> Or remind author if CI failed?
>>>>>> 
>>>>>> Does it make sence?
>>>>>> 
>>>>>> 
>>>>>> Best wish.
>>>>>> -- Jiajie
>>>>>> ________________________________
>>>>>> From: airflowuser <airflowuser@protonmail.com.INVALID>
>>>>>> Sent: Tuesday, April 23, 2019 16:39
>>>>>> To: dev@airflow.apache.org
>>>>>> Subject: Re: Proposal: Automatically mark stale PRs in github
>>>>>> 
>>>>>> Since there are many many open PRs in the repo it can be hard for
>>>>>> committers to keep track (I think that you are keeping tack by the
>>>>> mailing
>>>>>> list which sometimes can easily be missed).
>>>>>> 
>>>>>> It may be easier to tack using the filter of recently updated (see
>>>>> image)
>>>>>> I hoped that some day this will be the default order of PRs. That
way
>>>>>> activity in a PR from the last page would bump it to the front.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Sent with ProtonMail Secure Email.
>>>>>> 
>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>>>> On Tuesday, April 23, 2019 11:32 AM, Ash Berlin-Taylor <
>>>>> ash@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>> As a user/reporter on other opensource projects I would personally
>>>>> see
>>>>>> auto-close after 30 days to be far too aggressive to the point of
>>>>> being
>>>>>> unfriendly to contributions.
>>>>>>> 
>>>>>>> Unless we get markedly better at merging PRs I wouldn't want
to see
>>>>> us
>>>>>> mark as stale so quickly.
>>>>>>> 
>>>>>>> -ash
>>>>>>> 
>>>>>>>> On 22 Apr 2019, at 22:07, Jarek Potiuk Jarek.Potiuk@polidea.com
>>>>> wrote:
>>>>>>>> Here is a better search showing all the 103 issues - all
of them
>>>>>> "updated"
>>>>>>>> 17 days ago
>>>>>>>> 
>>>>>> 
>>>>> https://github.com/apache/airflow/pulls?page=1&q=is%3Apr+is%3Aopen+updated%3A
>>>>>> <2019-04-06+sort%3Aupdated-desc
>>>>>>>> On Mon, Apr 22, 2019 at 11:06 PM Jarek Potiuk
>>>>> Jarek.Potiuk@polidea.com
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> I think current stalebot configuration will not help
us for
>>>>> quite a
>>>>>> while
>>>>>>>>> for mysterious reason.
>>>>>>>>> I looked at the current PRs and somehow mysteriously
vast
>>>>> majority of
>>>>>>>>> issues (even issues last-commented in 2017) have been
updated 17
>>>>>> days ago.
>>>>>>>>> 
>>>>>> 
>>>>> https://drive.google.com/file/d/19GF1fdpYa2Tf25N3XgAEKrdXBwr9mNH9/view?usp=sharing
>>>>>>>>> It looks like they were all updated on 6th of April,
at 00:13
>>>>> CEST.
>>>>>>>>> There are 103 such issues:
>>>>>>>>> 
>>>>>> 
>>>>> https://github.com/apache/airflow/pulls?utf8=✓&q=is%3Apr+is%3Aopen+updated%3A
>>>>> <https://github.com/apache/airflow/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+updated%3A>
>>>>>> <
>>>>> https://github.com/apache/airflow/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+updated%3A
>>>>>> 
>>>>>> <2019-04-06+.
>>>>>>>>> It would be nice to find out why this happened.
>>>>>>>>> From stalebot documentation: "Any change to an issues
and pull
>>>>>> request is
>>>>>>>>> considered an update, including comments, changing labels,
>>>>> applying
>>>>>> or
>>>>>>>>> removing milestones, or pushing commits.". I think none
of that
>>>>>> happened to
>>>>>>>>> most of the 103 issues (i checked a few and could not
find any
>>>>> trace
>>>>>> of any
>>>>>>>>> such changes). But maybe someone can recall something
that
>>>>> happened
>>>>>> 6th of
>>>>>>>>> April around midnight (Saturday).
>>>>>>>>> Current configuration of stalebot (.github/stalebot.yaml)
says:
>>>>> 45
>>>>>> days
>>>>>>>>> (mark as stakle) and further 7 days (closing). So those
issues
>>>>> will
>>>>>> be
>>>>>>>>> marked as stale by the stalebot around May 20th (providing
that
>>>>> such
>>>>>> update
>>>>>>>>> won't happen again).
>>>>>>>>> Maybe then we can set it to 20 days + 7 for now to stale
most
>>>>> issues
>>>>>> up
>>>>>>>>> in 3 days and delete them 10 days from now? If the config
will
>>>>> be too
>>>>>>>>> aggressive we can change it back after the 103 issues
are
>>>>> cleaned-up.
>>>>>>>>> J.
>>>>>>>>> On Thu, Apr 18, 2019 at 7:54 AM airflowuser
>>>>>>>>> airflowuser@protonmail.com.invalid wrote:
>>>>>>>>> 
>>>>>>>>>> It's already on (or at least was on in December 2018).
>>>>>>>>>> In any case here is a list of old PRs that are waiting
for
>>>>>> committers.
>>>>>>>>>> [AIRFLOW-1956] Add parameter whether the navbar clock
time is
>>>>> UTC
>>>>>>>>>> https://github.com/apache/airflow/pull/2906
>>>>>>>>>> Status: ash commented but there are no further instructions.
>>>>>>>>>> [AIRFLOW-620] Feature to tail custom number of logs
instead of
>>>>>> rendering
>>>>>>>>>> whole log
>>>>>>>>>> https://github.com/apache/airflow/pull/3992
>>>>>>>>>> Status: Pushed changed in Jan 2019 that were not
reviewed
>>>>>>>>>> AIRFLOW-3149 Support dataproc cluster deletion on
ERROR
>>>>>>>>>> https://github.com/apache/airflow/pull/4064
>>>>>>>>>> Status: pushed changes today. CI passed.
>>>>>>>>>> [AIRFLOW-1424] make the next execution date of DAGs
visible
>>>>>>>>>> https://github.com/apache/airflow/pull/2460
>>>>>>>>>> Status: not sure. Waiting for ash ?
>>>>>>>>>> [AIRFLOW-1488] Add the TriggeredDagRunSensor operator
>>>>>>>>>> https://github.com/apache/airflow/pull/4291
>>>>>>>>>> Status: Waiting for code review
>>>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>>>>>>>> On Thursday, April 18, 2019 12:01 AM, Daniel Imberman
<
>>>>>>>>>> dimberman.opensource@gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> As part of our effort to reduce the PR backlog
I wanted to
>>>>>> proposed that
>>>>>>>>>>> we set the github stale action
>>>>> https://github.com/apps/stale.
>>>>>> This will
>>>>>>>>>>> allow us to temporarily close PRs/tickets that
are not
>>>>> actively
>>>>>> being
>>>>>>>>>>> worked on.
>>>>>>>>>>> (note that this will not remove PRs, it will
simply mark
>>>>> PRs as
>>>>>> stale to
>>>>>>>>>>> make it easier for committers)
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Jarek Potiuk
>>>>>>>>> Polidea https://www.polidea.com/ | Principal Software
Engineer
>>>>>>>>> M: +48 660 796 129 <+48660796129>
>>>>>>>>> E: jarek.potiuk@polidea.com
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Jarek Potiuk
>>>>>>>> Polidea https://www.polidea.com/ | Principal Software Engineer
>>>>>>>> M: +48 660 796 129 <+48660796129>
>>>>>>>> E: jarek.potiuk@polidea.com
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> --
>>>>> 
>>>>> Jarek Potiuk
>>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>>> 
>>>>> M: +48 660 796 129 <+48660796129>
>>>>> E: jarek.potiuk@polidea.com
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> 
>>>> Jarek Potiuk
>>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>>> 
>>>> M: +48 660 796 129 <+48660796129>
>>>> E: jarek.potiuk@polidea.com
>>>> 
>>> 
>>> 
>>> --
>>> 
>>> Jarek Potiuk
>>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>> 
>>> M: +48 660 796 129 <+48660796129>
>>> E: jarek.potiuk@polidea.com
>>> 
>> 
>> 
>> --
>> 
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>> 
>> M: +48 660 796 129 <+48660796129>
>> E: jarek.potiuk@polidea.com
>> 
> 
> 
> -- 
> 
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
> 
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>


Mime
View raw message