From dev-return-8806-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Mon Jul 1 09:55:17 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id B83A7180670 for ; Mon, 1 Jul 2019 11:55:16 +0200 (CEST) Received: (qmail 70019 invoked by uid 500); 1 Jul 2019 09:55:11 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 70004 invoked by uid 99); 1 Jul 2019 09:55:11 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jul 2019 09:55:11 +0000 Received: from themisto-2.localdomain (231.25.169.217.in-addr.arpa [217.169.25.231]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 4AF5F813F for ; Mon, 1 Jul 2019 09:55:07 +0000 (UTC) From: Ash Berlin-Taylor Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: Proposal: Automatically mark stale PRs in github Date: Mon, 1 Jul 2019 10:55:05 +0100 References: <32AB68C4-DF52-44AB-9A17-720059F0F148@apache.org> To: dev@airflow.apache.org In-Reply-To: Message-Id: <9AD6E95C-B46F-4479-AC6A-32E19F6A8134@apache.org> X-Mailer: Apple Mail (2.3445.104.11) An example of why I'm not a _huge_ van of stale bot, at least not for = issues. https://github.com/dpgaspar/Flask-AppBuilder/issues/685 That is still an issue but was closed just because no one responded to = it. > On 11 Jun 2019, at 06:50, Jarek Potiuk = wrote: >=20 > This issue bugs me a lot. Pretty much all our PRs were updated 2 days = ago > again :( >=20 > I've opened the ticket to Apache Infrastructure > https://issues.apache.org/jira/browse/INFRA-18589 and I hope we can = get to > the bottom of it. I believe it might be some integration we have (but = I > have no access to it). I looked at other Apache repositories and they = do > not have similar "updates" happening, so it must be something specific = for > apache/airflow repo. >=20 > J. >=20 > On Thu, May 30, 2019 at 10:41 PM Jarek Potiuk = > wrote: >=20 >> Well. Github support is quite far from being helpful :(. We'll have = to dig >> deeper on our own it seems >>=20 >> Our apologies for the wait, and thank you for getting in touch! Due = to a >> high volume of requests, we are currently experiencing much longer = than >> average response times here in Support. You asked: >>=20 >> Can you please let us know what action caused the update and what can = we >> do to prevent it from happening again ? >>=20 >> The updated_at for any object, including users, will change whenever = the >> database record for that object is updated. Such database updates can >> happen for many reasons, though we don't have a complete list of = those to >> share with you and your team. We wish could be of more help here as = we see >> how this can be a problem for you and your team, but we don't = currently >> have any other insight to share at this time. >>=20 >> Please let us know how else we can be of help! >>=20 >> On Sun, May 19, 2019 at 1:14 PM Jarek Potiuk = >> wrote: >>=20 >>> All our PRs were updated again on Wednesday, 15th of May. I am = following >>> up with Github support (they have not responded so far). >>>=20 >>> Maybe someone happens to know what could have caused the update = (some >>> automated job? bot? CI?). There is absolutely no update visible in = the UI >>> of github for those. I also looked at the fork in some cases - = nothing >>> changed for those either. >>>=20 >>> Or maybe someone has contact at Github so that they verify/fix it = faster >>> ? They must be able to see from the logs what happened to those PRs = - from >>> our point of view looks like most of those PRs were not touched for = several >>> months. >>> I responded to them with this (the ticket number is 159141). >>>=20 >>>=20 >>> Hello GitHub support, >>>=20 >>> We continue to have the same problem. Pretty much all our PR were = updated >>> again 4 days ago - which prevents stalebot from closing them. >>>=20 >>> Example here: https://github.com/apache/airflow/pull/4635 - this PR = was >>> last touched 3 months ago, yet when we list it with this query = https:// >>> github >>> = .com/apache/airflow/pulls?page=3D5&q=3Dis%3Apr+is%3Aopen+updated%3A%3C2019= -05-16+sort%3Aupdated-desc&utf8=3D%E2%9C%93 it >>> shows as updated 4 days ago (i.e. on Wed 15th of May). I cannot find = any >>> indicatio of a change that could have caused the update date to be = bumped >>> again. >>>=20 >>> Can you please let us know what action caused the update and what = can we >>> do to prevent it from happening again ? >>>=20 >>> J. >>>=20 >>>=20 >>> On Mon, May 6, 2019 at 3:54 PM Jarek Potiuk = >>> wrote: >>>=20 >>>> I raised an issue with Github. Let's see what they say: >>>>=20 >>>> Jarek, >>>>=20 >>>> Thank you for contacting GitHub Developer Support. We wanted to let = you >>>> know that we've received your message and will get to it as quickly = as >>>> possible. >>>>=20 >>>> Ticket ID: 159141 >>>>=20 >>>> We've also included a copy of your message below. >>>>=20 >>>> If you have any additional information or would like to add = anything to >>>> your initial message, now would be a great time to do so, feel free = to >>>> reply to this email. If not, then rest assured your request is in = the right >>>> hands :) >>>>=20 >>>> Thank you! >>>> The GitHub Developer Support Team >>>>=20 >>>> *Jarek Potiuk* >>>>=20 >>>> May 6, 1:47 PM UTC >>>>=20 >>>> Hello All, >>>>=20 >>>> In Apache Airflow project we are trying to use stalebot to closed >>>> not-updated pull requests. And for some reason the bot does not = really >>>> closed our old tickets. We checked what could be wrong and it seems = that >>>> pretty much all our PRs get somehow updated regularly. >>>>=20 >>>> Last time I checked more than 100 PRs were updated at 27th of April = and >>>> yesterday I checked that 118 requests were updated on 28th of = April. It >>>> does not seem that there was any action that could have caused the = updates. >>>>=20 >>>> Here are all the requests (all of them updated 27th of April): >>>>=20 >>>>=20 >>>> = https://github.com/apache/airflow/pulls?utf8=3D%E2%9C%93&q=3Dis%3Apr+is%3A= open+updated%3A%3C2019-04-28+sort%3Aupdated-desc+ >>>>=20 >>>> And here is an example PR that was updated 27th of April but there = seem >>>> to be no action that could have caused it: >>>> https://github.com/apache/airflow/pull/4929 >>>>=20 >>>> Can you please explain where the updates are coming from and how we = can >>>> avoid the updates from happening? >>>>=20 >>>> On Mon, May 6, 2019 at 3:39 AM Jiajie Zhong = >>>> wrote: >>>>=20 >>>>> It's really odd. I don't know this issue. I think maybe travis-c = update >>>>> our PR time at first but it don't. >>>>>=20 >>>>> BTW, I take a look on some PR and give some example. >>>>>=20 >>>>> https://github.com/apache/airflow/pull/5135 create 17 days ago, = last >>>>> comment 16 days ago, and travis-ci finish 17 days ago (which mean = that CI >>>>> process don't touch it and change PR update time) >>>>>=20 >>>>> https://github.com/apache/airflow/pull/5136 >>>>>=20 >>>>>=20 >>>>> Best wish. >>>>> -- Jiajie >>>>> ________________________________ >>>>> From: Jarek Potiuk >>>>> Sent: Monday, May 6, 2019 4:04 >>>>> To: dev@airflow.apache.org >>>>> Cc: airflowuser >>>>> Subject: Re: Proposal: Automatically mark stale PRs in github >>>>>=20 >>>>> I believe our current stale bot configuration does not work. And I = do >>>>> not >>>>> know the reason yet, which worries me :( >>>>>=20 >>>>> There is something really strange going on with our PRs and their >>>>> updated >>>>> date. Again pretty much all the PRs were mysteriously updated on = *27th >>>>> of >>>>> April - 8 days ago* (similarly as the previous case where I saw = all PRs >>>>> updated on *6th of April*). >>>>>=20 >>>>> You can see it here: >>>>>=20 >>>>> * there are just 2(!) PRs updated before 27th of April: >>>>>=20 >>>>> = https://github.com/apache/airflow/pulls?utf8=3D%E2%9C%93&q=3Dis%3Apr+is%3A= open+updated%3A%3C2019-04-27+sort%3Aupdated-desc+ >>>>> * there are 120 (!) PRS updated before 28th of April: >>>>>=20 >>>>> = https://github.com/apache/airflow/pulls?utf8=3D%E2%9C%93&q=3Dis%3Apr+is%3A= open+updated%3A%3C2019-04-28+sort%3Aupdated-desc+ >>>>>=20 >>>>> There is no indication that most of those impacted issues were at = all >>>>> touched on 27th or 28th of April. If you look at random PRs there, = most >>>>> of >>>>> them were commented latest at the beginning of April. >>>>>=20 >>>>> Looks like 8 days ago some process has bumped the update date for = most >>>>> of >>>>> our PRs. With this kind of "regular" (it seems) process of marking = the >>>>> requests "updated" our stale bot is useless. >>>>>=20 >>>>> Does anyone have an idea why it might have happened? >>>>>=20 >>>>> I am quite puzzled by this one. I am going to open an issue to = Github >>>>> support if no one has an idea what's going on. >>>>>=20 >>>>> J. >>>>>=20 >>>>>=20 >>>>>=20 >>>>> On Tue, Apr 23, 2019 at 12:39 PM Jiajie Zhong < >>>>> zhongjiajie955@hotmail.com> >>>>> wrote: >>>>>=20 >>>>>> I think we should change stale-bot strategy to auto close PR, If = 30 >>>>> days >>>>>> is too short for contributions, is 60 or 90 days make sence? >>>>>>=20 >>>>>> In addition, I notice that we have some PR pass CI but none = review it >>>>> or >>>>>> let a suggest on it. So could we add a bot auto remind committer = if >>>>> PR pass >>>>>> CI but no one review? >>>>>>=20 >>>>>> Or remind author if CI failed? >>>>>>=20 >>>>>> Does it make sence? >>>>>>=20 >>>>>>=20 >>>>>> Best wish. >>>>>> -- Jiajie >>>>>> ________________________________ >>>>>> From: airflowuser >>>>>> Sent: Tuesday, April 23, 2019 16:39 >>>>>> To: dev@airflow.apache.org >>>>>> Subject: Re: Proposal: Automatically mark stale PRs in github >>>>>>=20 >>>>>> Since there are many many open PRs in the repo it can be hard for >>>>>> committers to keep track (I think that you are keeping tack by = the >>>>> mailing >>>>>> list which sometimes can easily be missed). >>>>>>=20 >>>>>> It may be easier to tack using the filter of recently updated = (see >>>>> image) >>>>>> I hoped that some day this will be the default order of PRs. That = way >>>>>> activity in a PR from the last page would bump it to the front. >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> Sent with ProtonMail Secure Email. >>>>>>=20 >>>>>> =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 = Original Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2= =80=90 >>>>>> On Tuesday, April 23, 2019 11:32 AM, Ash Berlin-Taylor < >>>>> ash@apache.org> >>>>>> wrote: >>>>>>=20 >>>>>>> As a user/reporter on other opensource projects I would = personally >>>>> see >>>>>> auto-close after 30 days to be far too aggressive to the point of >>>>> being >>>>>> unfriendly to contributions. >>>>>>>=20 >>>>>>> Unless we get markedly better at merging PRs I wouldn't want to = see >>>>> us >>>>>> mark as stale so quickly. >>>>>>>=20 >>>>>>> -ash >>>>>>>=20 >>>>>>>> On 22 Apr 2019, at 22:07, Jarek Potiuk Jarek.Potiuk@polidea.com >>>>> wrote: >>>>>>>> Here is a better search showing all the 103 issues - all of = them >>>>>> "updated" >>>>>>>> 17 days ago >>>>>>>>=20 >>>>>>=20 >>>>> = https://github.com/apache/airflow/pulls?page=3D1&q=3Dis%3Apr+is%3Aopen+upd= ated%3A >>>>>> <2019-04-06+sort%3Aupdated-desc >>>>>>>> On Mon, Apr 22, 2019 at 11:06 PM Jarek Potiuk >>>>> Jarek.Potiuk@polidea.com >>>>>>>> wrote: >>>>>>>>=20 >>>>>>>>> I think current stalebot configuration will not help us for >>>>> quite a >>>>>> while >>>>>>>>> for mysterious reason. >>>>>>>>> I looked at the current PRs and somehow mysteriously vast >>>>> majority of >>>>>>>>> issues (even issues last-commented in 2017) have been updated = 17 >>>>>> days ago. >>>>>>>>>=20 >>>>>>=20 >>>>> = https://drive.google.com/file/d/19GF1fdpYa2Tf25N3XgAEKrdXBwr9mNH9/view?usp= =3Dsharing >>>>>>>>> It looks like they were all updated on 6th of April, at 00:13 >>>>> CEST. >>>>>>>>> There are 103 such issues: >>>>>>>>>=20 >>>>>>=20 >>>>> = https://github.com/apache/airflow/pulls?utf8=3D=E2=9C=93&q=3Dis%3Apr+is%3A= open+updated%3A >>>>> = >>>>>> < >>>>> = https://github.com/apache/airflow/pulls?utf8=3D%E2%9C%93&q=3Dis%3Apr+is%3A= open+updated%3A >>>>>>=20 >>>>>> <2019-04-06+. >>>>>>>>> It would be nice to find out why this happened. >>>>>>>>> =46rom stalebot documentation: "Any change to an issues and = pull >>>>>> request is >>>>>>>>> considered an update, including comments, changing labels, >>>>> applying >>>>>> or >>>>>>>>> removing milestones, or pushing commits.". I think none of = that >>>>>> happened to >>>>>>>>> most of the 103 issues (i checked a few and could not find any >>>>> trace >>>>>> of any >>>>>>>>> such changes). But maybe someone can recall something that >>>>> happened >>>>>> 6th of >>>>>>>>> April around midnight (Saturday). >>>>>>>>> Current configuration of stalebot (.github/stalebot.yaml) = says: >>>>> 45 >>>>>> days >>>>>>>>> (mark as stakle) and further 7 days (closing). So those issues >>>>> will >>>>>> be >>>>>>>>> marked as stale by the stalebot around May 20th (providing = that >>>>> such >>>>>> update >>>>>>>>> won't happen again). >>>>>>>>> Maybe then we can set it to 20 days + 7 for now to stale most >>>>> issues >>>>>> up >>>>>>>>> in 3 days and delete them 10 days from now? If the config will >>>>> be too >>>>>>>>> aggressive we can change it back after the 103 issues are >>>>> cleaned-up. >>>>>>>>> J. >>>>>>>>> On Thu, Apr 18, 2019 at 7:54 AM airflowuser >>>>>>>>> airflowuser@protonmail.com.invalid wrote: >>>>>>>>>=20 >>>>>>>>>> It's already on (or at least was on in December 2018). >>>>>>>>>> In any case here is a list of old PRs that are waiting for >>>>>> committers. >>>>>>>>>> [AIRFLOW-1956] Add parameter whether the navbar clock time is >>>>> UTC >>>>>>>>>> https://github.com/apache/airflow/pull/2906 >>>>>>>>>> Status: ash commented but there are no further instructions. >>>>>>>>>> [AIRFLOW-620] Feature to tail custom number of logs instead = of >>>>>> rendering >>>>>>>>>> whole log >>>>>>>>>> https://github.com/apache/airflow/pull/3992 >>>>>>>>>> Status: Pushed changed in Jan 2019 that were not reviewed >>>>>>>>>> AIRFLOW-3149 Support dataproc cluster deletion on ERROR >>>>>>>>>> https://github.com/apache/airflow/pull/4064 >>>>>>>>>> Status: pushed changes today. CI passed. >>>>>>>>>> [AIRFLOW-1424] make the next execution date of DAGs visible >>>>>>>>>> https://github.com/apache/airflow/pull/2460 >>>>>>>>>> Status: not sure. Waiting for ash ? >>>>>>>>>> [AIRFLOW-1488] Add the TriggeredDagRunSensor operator >>>>>>>>>> https://github.com/apache/airflow/pull/4291 >>>>>>>>>> Status: Waiting for code review >>>>>>>>>> =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90= Original Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2= =80=90 >>>>>>>>>> On Thursday, April 18, 2019 12:01 AM, Daniel Imberman < >>>>>>>>>> dimberman.opensource@gmail.com> wrote: >>>>>>>>>>=20 >>>>>>>>>>> As part of our effort to reduce the PR backlog I wanted to >>>>>> proposed that >>>>>>>>>>> we set the github stale action >>>>> https://github.com/apps/stale. >>>>>> This will >>>>>>>>>>> allow us to temporarily close PRs/tickets that are not >>>>> actively >>>>>> being >>>>>>>>>>> worked on. >>>>>>>>>>> (note that this will not remove PRs, it will simply mark >>>>> PRs as >>>>>> stale to >>>>>>>>>>> make it easier for committers) >>>>>>>>>=20 >>>>>>>>> -- >>>>>>>>> Jarek Potiuk >>>>>>>>> Polidea https://www.polidea.com/ | Principal Software Engineer >>>>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>>>> E: jarek.potiuk@polidea.com >>>>>>>>=20 >>>>>>>> -- >>>>>>>> Jarek Potiuk >>>>>>>> Polidea https://www.polidea.com/ | Principal Software Engineer >>>>>>>> M: +48 660 796 129 <+48660796129> >>>>>>>> E: jarek.potiuk@polidea.com >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>> -- >>>>>=20 >>>>> Jarek Potiuk >>>>> Polidea | Principal Software Engineer >>>>>=20 >>>>> M: +48 660 796 129 <+48660796129> >>>>> E: jarek.potiuk@polidea.com >>>>>=20 >>>>=20 >>>>=20 >>>> -- >>>>=20 >>>> Jarek Potiuk >>>> Polidea | Principal Software Engineer >>>>=20 >>>> M: +48 660 796 129 <+48660796129> >>>> E: jarek.potiuk@polidea.com >>>>=20 >>>=20 >>>=20 >>> -- >>>=20 >>> Jarek Potiuk >>> Polidea | Principal Software Engineer >>>=20 >>> M: +48 660 796 129 <+48660796129> >>> E: jarek.potiuk@polidea.com >>>=20 >>=20 >>=20 >> -- >>=20 >> Jarek Potiuk >> Polidea | Principal Software Engineer >>=20 >> M: +48 660 796 129 <+48660796129> >> E: jarek.potiuk@polidea.com >>=20 >=20 >=20 > --=20 >=20 > Jarek Potiuk > Polidea | Principal Software Engineer >=20 > M: +48 660 796 129 <+48660796129> > [image: Polidea]