climate-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John McGibbney <lewi...@apache.org>
Subject Re: [EXTERNAL] Re: [DISCUSS] Retire OCW?
Date Sat, 23 Nov 2019 17:20:21 GMT
I also think that these are excellent tasks.
A cleanup of JIRA and branching legacy/unused code would be an excellent start in order to
cleanup master.
Lewis

On 2019/11/21 11:09:53, Michael Anderson <michael.arthur.anderson@gmail.com> wrote:

> To expand on that thought:
> 
> Remove the UI.
> Remove anything related to virtualization and / or migrate it to AWS /
> Kubernetes.
> Remove any test or example that is not self contained (i.e. depends on
> inputs that are not publicly available).   Shrink the examples to a smaller
> number which exercise a larger number of features.
> Either move mccsearch to the examples or migrate the core functions to the
> core library.
> 
> Regarding the JIRA, I'd suggested that anything more than 2 years old can
> closed off.
> 
> I'd suggest this would make it both easier for people looking to contribute
> to find an impactful entry point and also simplify the migration path if
> it's to be consolidated into Alex's new library.
> 
> On Thu, Nov 21, 2019 at 5:25 AM Michael Anderson <
> michael.arthur.anderson@gmail.com> wrote:
> 
> > One of the challenges of working on the current project is that it isn't
> > always clear which of the sub folders are deprecated / some of the JIRA are
> > quite old and the original requestor is no longer interested.   I'd offer
> > that an aggressive pruning of the JIRA and sub folders would make it easier
> > for people to engage on features that would be most impactful and long
> > lived.   Barring that, that is what I find appealing about Alex's
> > suggestion as it clearly delineates what is no longer being supported and
> > gives a clearer roadmap on where to make the most impactful contributions.
> >
> > On Wed, Nov 20, 2019 at 4:49 PM Goodman, Alexander (US 398K)
> > <alexander.goodman@jpl.nasa.gov.invalid> wrote:
> >
> >> Hi Lewis,
> >>
> >> I can't say I completely disagree with your assessment. I would go even
> >> further and argue what you are saying has been true not just for 3 months
> >> but for at least since the 1.3.0 release, if not longer. But to be fair,
> >> most of the contributions to OCW including BCDP have been made possible by
> >> JPL/NASA funding our labor. Even though Kyo and I have had some success in
> >> obtaining a little bit more funding, it has been a very difficult path when
> >> we are the only primary developers, compared to the distant past when many
> >> more people were on board which is something I explain in the slides I sent
> >> as being one of the primary motivators for developing BCDP. The hope is
> >> that a more up to date API could alleviate some of the problems we have
> >> been having with maintaining the codebase since it's much smaller (xarray
> >> helps make many of the things we were previously doing more elegant and
> >> concise after all...), but it is still not quite at the stage where it can
> >> fully replace the current API, and I hope we can reach that final step with
> >> just a bit more funding.
> >>
> >> As someone with more knowledge about ASF protocol than I, I do think we
> >> should have a discussion about what our future software development
> >> practices should be, and we should definitely reach an agreement on what
> >> the best approach for adding BCDP to our repository is (and CMDA for that
> >> matter). I definitely don't want to just dump everything haphazardly into
> >> separate subfolders like we always do (which is something I mentioned in a
> >> recent JIRA issue thread that you may recall), but ideally we would like to
> >> at least keep the OCW brand alive. However if things continue to operate at
> >> the same sluggish pace and scale as they have been for sometime, then
> >> perhaps we should assess the merit of continuing to keep such a small
> >> project under the ASF's care. If we choose to declare the death of OCW, so
> >> be it, but I will still release BCDP as a separate project outside the ASF
> >> if I must.
> >>
> >> Kyo is on vacation for the rest of the week, but I am free to talk with
> >> you more about this offline if you are around this week since I'd like to
> >> have a very indepth discussion. Would you be interested?
> >>
> >> Thanks,
> >> Alex
> >> On 11/20/19, 12:25 PM, "Lewis John McGibbney" <lewismc@apache.org>
> >> wrote:
> >>
> >>     This gets at my original statement. To expand however, let me state,
> >> project activity is very low, new contributions are not being reviewed, the
> >> roadmap is not clear, there seems to be a proposal to literally drop-in
> >> replace current master OCW codebase with BCDP which has yet to be open
> >> sourced.
> >>     None of this is particularly pro-community growth.
> >>     I am not particularly happy with the way the community is being
> >> managed here at Apache. Nothing is particularly clear. The community seems
> >> very stagnant and pretty much dead. This is in agreement with
> >> reporter.apache.org community health score which is -3.47 and indicates
> >> that work is required.
> >>     This thread actually is the most activity the project has seen in the
> >> last 3 months!
> >>     I had to file the PMC report this month... for those who want to see
> >> what that looked like please monitor whimsy.
> >>     I think some timelines backing up Alex's proposals are needed.
> >> Otherwise we are just delaying the slow death of OCW further.
> >>     Lewis
> >>
> >>     On 2019/11/20 16:52:47, Michael Anderson <
> >> michael.arthur.anderson@gmail.com> wrote:
> >>     > That’s pretty interesting.   I like adding incremental features
> >> with the goal of replacing rather than outright demise of the old library.
> >>     >
> >>     > Sent from my iPhone
> >>     >
> >>     > > On Nov 20, 2019, at 11:08 AM, Goodman, Alexander (US 398K)
> >> <alexander.goodman@jpl.nasa.gov.invalid> wrote:
> >>     > >
> >>     > > Since the plots didn't attach in the last message, here they are:
> >>     > >
> >>     > > https://i.imgur.com/mAuq0R5.png
> >>     > > https://i.imgur.com/TeRYSPI.png
> >>     > >
> >>     > > Thanks,
> >>     > > Alex
> >>     > > -----Original Message-----
> >>     > > From: Goodman, Alexander (US 398K)
> >> <alexander.goodman@jpl.nasa.gov.INVALID>
> >>     > > Sent: Wednesday, November 20, 2019 8:05 AM
> >>     > > To: dev@climate.apache.org
> >>     > > Subject: RE: [EXTERNAL] Re: [DISCUSS] Retire OCW?
> >>     > >
> >>     > > Hi Michael,
> >>     > >
> >>     > > Please see this overview slide:
> >>     > > https://i.imgur.com/VnijQ5C.png
> >>     > >
> >>     > > Here is also a link of some older slides which have a lot more
> >> details:
> >>     > >
> >> https://docs.google.com/presentation/d/1nhD3_fZmVcmBnq9NiMBYeDMsxr5kIimI7dE9NNUQJEM/edit?usp=sharing
> >>     > >
> >>     > > I also created an end-to-end processing benchmark that I compared
> >> with the current OCW, which I have attached in two plots: One with
> >> regridding and one without, since the difference is so vast. I first
> >> presented these results at the AMS Annual meeting last year in Phoenix.
> >>     > >
> >>     > > Let me know if you have any questions.
> >>     > >
> >>     > > Thanks,
> >>     > > Alex
> >>     > > -----Original Message-----
> >>     > > From: Michael Anderson <michael.arthur.anderson@gmail.com>
> >>     > > Sent: Tuesday, November 19, 2019 5:59 PM
> >>     > > To: dev@climate.apache.org
> >>     > > Subject: [EXTERNAL] Re: [DISCUSS] Retire OCW?
> >>     > >
> >>     > > Do you have an overview of the new library you could share?  It'd
> >> be interesting how big the gap between the old and new would be / rough LOE
> >> to add parity.
> >>     > >
> >>     > >> On Tue, Nov 19, 2019 at 7:35 PM Alex Goodman <goodman@apache.org>
> >> wrote:
> >>     > >>
> >>     > >> Hi Lewis,
> >>     > >>
> >>     > >> To add some more specifics, the new xarray-based API I have
> >> developed
> >>     > >> for OCW called BCDP (Big Climate Data Pipeline) has recently
> >> been
> >>     > >> approved for release to open source. Keep in mind, this is
a
> >>     > >> completely new API that is not backwards compatible, so we
were
> >>     > >> planning to maintain both the old and new API under one
> >> repository
> >>     > >> (either as separate subfolders or git
> >>     > >> branches) until the new API has enough functionality to fully
> >> replace
> >>     > >> the old one. To accelerate this process and encourage more
> >>     > >> development, both Kyo and I are seeking some additional funding
> >>     > >> sources which we should be more certain about early next year.
> >> When I
> >>     > >> first conceived BCDP, I was originally planning on releasing
it
> >> to
> >>     > >> open source as a separate project outside of OCW, but after
some
> >>     > >> discussion we thought that it would be better to maintain
the
> >> OCW brand for it if possible.
> >>     > >>
> >>     > >> Does this make sense?
> >>     > >>
> >>     > >> Thanks,
> >>     > >> Alex
> >>     > >>> On 2019/11/20 00:17:22, lewis john mcgibbney <
> >> lewismc@apache.org> wrote:
> >>     > >>> I got this from Kyo offline
> >>     > >>>
> >>     > >>> “There is enough motivation to keep OCW going. Alex
has almost
> >>     > >>> refactored OCW and Seungwon Lee's CMDA will be a part
of OCW.”
> >>     > >>>
> >>     > >>> Some of us will remember that CMDA was retired from the
Apache
> >>     > >>> Incubator previously.
> >>     > >>>
> >>     > >>> It would be great if these plans could be hashed out on
the
> >> mailing
> >>     > >>> list
> >>     > >> so
> >>     > >>> we can move forward either way.
> >>     > >>>
> >>     > >>> Lewis
> >>     > >>>
> >>     > >>> On Thu, Nov 14, 2019 at 09:32 Lewis John Mcgibbney <
> >>     > >>> lewis.mcgibbney@gmail.com> wrote:
> >>     > >>>
> >>     > >>>> Hi Folks,
> >>     > >>>> Project activity is very low and new contributions
are not
> >> being
> >>     > >> reviewed.
> >>     > >>>> Simply, is it time to retire OCW to the Apache Attic?
> >>     > >>>> Best
> >>     > >>>> Lewis
> >>     > >>>>
> >>     > >>>> --
> >>     > >>>>
> >>     > >>>> *Lewis*
> >>     > >>>> Dr. Lewis J. McGibbney Ph.D, B.Sc
> >>     > >>>> *Skype*: lewis.john.mcgibbney
> >>     > >>>>
> >>     > >>>>
> >>     > >>>>
> >>     > >>>> --
> >>     > >>> http://home.apache.org/~lewismc/
> >>     > >>> http://people.apache.org/keys/committer/lewismc
> >>     > >>>
> >>     > >>
> >>     >
> >>
> >>
> >>
> 

Mime
View raw message