climate-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Goodman, Alexander (US 398K)" <alexander.good...@jpl.nasa.gov.INVALID>
Subject Re: [EXTERNAL] Re: [DISCUSS] Retire OCW?
Date Wed, 20 Nov 2019 21:49:06 GMT
Hi Lewis,

I can't say I completely disagree with your assessment. I would go even further and argue
what you are saying has been true not just for 3 months but for at least since the 1.3.0 release,
if not longer. But to be fair, most of the contributions to OCW including BCDP have been made
possible by JPL/NASA funding our labor. Even though Kyo and I have had some success in obtaining
a little bit more funding, it has been a very difficult path when we are the only primary
developers, compared to the distant past when many more people were on board which is something
I explain in the slides I sent as being one of the primary motivators for developing BCDP.
The hope is that a more up to date API could alleviate some of the problems we have been having
with maintaining the codebase since it's much smaller (xarray helps make many of the things
we were previously doing more elegant and concise after all...), but it is still not quite
at the stage where it can fully replace the current API, and I hope we can reach that final
step with just a bit more funding.

As someone with more knowledge about ASF protocol than I, I do think we should have a discussion
about what our future software development practices should be, and we should definitely reach
an agreement on what the best approach for adding BCDP to our repository is (and CMDA for
that matter). I definitely don't want to just dump everything haphazardly into separate subfolders
like we always do (which is something I mentioned in a recent JIRA issue thread that you may
recall), but ideally we would like to at least keep the OCW brand alive. However if things
continue to operate at the same sluggish pace and scale as they have been for sometime, then
perhaps we should assess the merit of continuing to keep such a small project under the ASF's
care. If we choose to declare the death of OCW, so be it, but I will still release BCDP as
a separate project outside the ASF if I must. 

Kyo is on vacation for the rest of the week, but I am free to talk with you more about this
offline if you are around this week since I'd like to have a very indepth discussion. Would
you be interested?

Thanks,
Alex
On 11/20/19, 12:25 PM, "Lewis John McGibbney" <lewismc@apache.org> wrote:

    This gets at my original statement. To expand however, let me state, project activity
is very low, new contributions are not being reviewed, the roadmap is not clear, there seems
to be a proposal to literally drop-in replace current master OCW codebase with BCDP which
has yet to be open sourced. 
    None of this is particularly pro-community growth.
    I am not particularly happy with the way the community is being managed here at Apache.
Nothing is particularly clear. The community seems very stagnant and pretty much dead. This
is in agreement with  reporter.apache.org community health score which is -3.47 and indicates
that work is required. 
    This thread actually is the most activity the project has seen in the last 3 months!
    I had to file the PMC report this month... for those who want to see what that looked
like please monitor whimsy.
    I think some timelines backing up Alex's proposals are needed. Otherwise we are just delaying
the slow death of OCW further.
    Lewis
    
    On 2019/11/20 16:52:47, Michael Anderson <michael.arthur.anderson@gmail.com> wrote:

    > That’s pretty interesting.   I like adding incremental features with the goal of
replacing rather than outright demise of the old library.  
    > 
    > Sent from my iPhone
    > 
    > > On Nov 20, 2019, at 11:08 AM, Goodman, Alexander (US 398K) <alexander.goodman@jpl.nasa.gov.invalid>
wrote:
    > > 
    > > Since the plots didn't attach in the last message, here they are:
    > > 
    > > https://i.imgur.com/mAuq0R5.png
    > > https://i.imgur.com/TeRYSPI.png
    > > 
    > > Thanks,
    > > Alex
    > > -----Original Message-----
    > > From: Goodman, Alexander (US 398K) <alexander.goodman@jpl.nasa.gov.INVALID>

    > > Sent: Wednesday, November 20, 2019 8:05 AM
    > > To: dev@climate.apache.org
    > > Subject: RE: [EXTERNAL] Re: [DISCUSS] Retire OCW?
    > > 
    > > Hi Michael,
    > > 
    > > Please see this overview slide:
    > > https://i.imgur.com/VnijQ5C.png
    > > 
    > > Here is also a link of some older slides which have a lot more details:
    > > https://docs.google.com/presentation/d/1nhD3_fZmVcmBnq9NiMBYeDMsxr5kIimI7dE9NNUQJEM/edit?usp=sharing
    > > 
    > > I also created an end-to-end processing benchmark that I compared with the current
OCW, which I have attached in two plots: One with regridding and one without, since the difference
is so vast. I first presented these results at the AMS Annual meeting last year in Phoenix.
    > > 
    > > Let me know if you have any questions.
    > > 
    > > Thanks,
    > > Alex
    > > -----Original Message-----
    > > From: Michael Anderson <michael.arthur.anderson@gmail.com>
    > > Sent: Tuesday, November 19, 2019 5:59 PM
    > > To: dev@climate.apache.org
    > > Subject: [EXTERNAL] Re: [DISCUSS] Retire OCW?
    > > 
    > > Do you have an overview of the new library you could share?  It'd be interesting
how big the gap between the old and new would be / rough LOE to add parity.
    > > 
    > >> On Tue, Nov 19, 2019 at 7:35 PM Alex Goodman <goodman@apache.org>
wrote:
    > >> 
    > >> Hi Lewis,
    > >> 
    > >> To add some more specifics, the new xarray-based API I have developed 
    > >> for OCW called BCDP (Big Climate Data Pipeline) has recently been 
    > >> approved for release to open source. Keep in mind, this is a 
    > >> completely new API that is not backwards compatible, so we were 
    > >> planning to maintain both the old and new API under one repository 
    > >> (either as separate subfolders or git
    > >> branches) until the new API has enough functionality to fully replace 
    > >> the old one. To accelerate this process and encourage more 
    > >> development, both Kyo and I are seeking some additional funding 
    > >> sources which we should be more certain about early next year. When I 
    > >> first conceived BCDP, I was originally planning on releasing it to 
    > >> open source as a separate project outside of OCW, but after some 
    > >> discussion we thought that it would be better to maintain the OCW brand
for it if possible.
    > >> 
    > >> Does this make sense?
    > >> 
    > >> Thanks,
    > >> Alex
    > >>> On 2019/11/20 00:17:22, lewis john mcgibbney <lewismc@apache.org>
wrote:
    > >>> I got this from Kyo offline
    > >>> 
    > >>> “There is enough motivation to keep OCW going. Alex has almost 
    > >>> refactored OCW and Seungwon Lee's CMDA will be a part of OCW.”
    > >>> 
    > >>> Some of us will remember that CMDA was retired from the Apache 
    > >>> Incubator previously.
    > >>> 
    > >>> It would be great if these plans could be hashed out on the mailing

    > >>> list
    > >> so
    > >>> we can move forward either way.
    > >>> 
    > >>> Lewis
    > >>> 
    > >>> On Thu, Nov 14, 2019 at 09:32 Lewis John Mcgibbney < 
    > >>> lewis.mcgibbney@gmail.com> wrote:
    > >>> 
    > >>>> Hi Folks,
    > >>>> Project activity is very low and new contributions are not being
    > >> reviewed.
    > >>>> Simply, is it time to retire OCW to the Apache Attic?
    > >>>> Best
    > >>>> Lewis
    > >>>> 
    > >>>> --
    > >>>> 
    > >>>> *Lewis*
    > >>>> Dr. Lewis J. McGibbney Ph.D, B.Sc
    > >>>> *Skype*: lewis.john.mcgibbney
    > >>>> 
    > >>>> 
    > >>>> 
    > >>>> --
    > >>> http://home.apache.org/~lewismc/
    > >>> http://people.apache.org/keys/committer/lewismc
    > >>> 
    > >> 
    > 
    

Mime
View raw message