community-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rich Bowen <rbo...@rcbowen.com>
Subject Re: Does GSoC help develop communities?
Date Tue, 06 Dec 2016 11:33:12 GMT
Ok, skip it then. However, I'm concerned that what we don't measure, we
don't know how to improve.

On Dec 6, 2016 3:36 AM, "Ulrich Stärk" <uli@spielviel.de> wrote:

> On 05.12.16 22:24, Rich Bowen wrote:
> > So ... stepping back a bit here. Are you saying that even attempting to
> > measure outcomes is harmful, because we might draw the wrong conclusions?
> >
> > This is a completely new notion to me.
> >
> > You say:
> >
> > "I am reluctant to simply collect some data because I am missing a clear
> > question what we are trying to answer and how the data you want to
> > collect is actually answering that question."
> >
> > I do have a clear question. My question is whether participating in the
> > GSoC contributes to our goal of Community Development. Does it develop
> > our community in any measurable way? I assume, at this point, that it
> > does, or we wouldn't keep doing it. In what way has it had measurable
> > impact on our community?
> >
> > Things that we can clearly measure is adding participants in our project
> > communities, and artifacts added to our projects' revision control
> > repositories. Measuring other things is more difficult, and can be taken
> > as a later task. Easy things first.
>
> And here is what I'm afraid of: By focusing on those two simple dimensions
> only we will from my
> experience end up with a pretty skewed result. The dimensions you propose
> don't cover code living on
> outside our repositories as add-on modules, they don't cover indirect
> contributions, etc.
>
> >
> > The larger question of how and what we measure in the Community
> > Development PMC as a whole hinges on this, too. I'm curious if you're
> > resistant to that, too?
>
> I am resistant to what you are proposing to measure. I am resistant to
> simplifying metrics. Those
> tend to do more harm than good.
>
> Uli
>
> >
> > --Rich
> >
> >
> >
> > On 12/05/2016 04:00 PM, Ulrich Stärk wrote:
> >> On 05.12.16 17:54, Rich Bowen wrote:
> >>>
> >>>
> >>> On 12/05/2016 07:41 AM, Ulrich Stärk wrote:
> >>>>> Or, at the very least, can we make a commitment to track this data
> going
> >>>>>> forward?
> >>>> Let me play the devil's advocate here: What for?
> >>>>
> >>>> GSoC is completely free for the ASF (on the contrary, we even get a
> small amount for every accepted
> >>>> student that we can than put towards fulfilling our goals) and as
> long as we have volunteers willing
> >>>> to organize it and mentor students we can assume that at least those
> volunteers are seeing value in
> >>>> it. Why the stats other than for satisfying our curiosity?
> >>>
> >>> Or, perhaps, let me give a different answer.
> >>>
> >>> I participated in GSoC as a mentor for $WorkProject. While it didn't
> >>> "cost" me anything in dollars, it cost me probably 200 hours of my
> time.
> >>> I know that other projects at work put more time in, and some less.
> >>>
> >>> This is an enormous cost to me, as an employee. So it behooves me to
> >>> measure the benefit to the project. A student received payment to write
> >>> code that was discarded. And I (and several of my colleagues) spent a
> >>> huge amount of time, which I could have spent on other things,
> mentoring
> >>> that student. Benefit to project, pretty heavily negative.
> >>
> >> In that case you should have failed the student pretty early as I tell
> mentors every year and as
> >> documented in the mentor guide.
> >>
> >>>
> >>> So, now, here we are at the ASF, doing GSoC with our projects, and
> >>> promoting it to them as a benefit. Does it actually benefit them, or is
> >>> it merely siphoning off time that could be spent on other things.
> >>
> >> If a mentor feels it is the latter, they should immediately fail the
> student.
> >>
> >>>
> >>> To folks that say we can't measure that, I strongly disagree. There are
> >>> two measures that are obvious and easy.
> >>>
> >>> 1) What % of GSoC student are still active on the project 6 months, 12
> >>> months, 18 months after the project. (We can debate the definition of
> >>> "active" all you like.
> >>
> >> This implies that the program is only successful if a certain number of
> students sticks around. And
> >> this is exactly what I'm arguing against. IMO the program is successful
> as soon as a student has
> >> some exposure to one of our communities.
> >>
> >>>
> >>> 2) What % of code developed by GSoC students actually becomes a part of
> >>> the project codebase at the end of the project?
> >>
> >> This has the same problems as trying to measure productivity from code.
> What is the percentage
> >> telling us about the question we want to answer? Also see above: If the
> code cannot be part of the
> >> project codebase the student should be failed.
> >>
> >>>
> >>> I would maintain that #1 is part of our charter as ComDev, and #2 is
> >>> part of what projects should be made aware of before they sign on.
> >>>
> >>> Again, I'm really not asking for a lot of data here. But I do think
> that
> >>> it's part of a responsible accounting for participating in GSoC. If,
> for
> >>> example, it is actually hindering projects, don't we want to know that.
> >>>
> >>> I did *not* participate in GSoC again, on $WorkProject, because it
> >>> clearly hindered my project.
> >>>
> >>>
> >>
> >> I am reluctant to simply collect some data because I am missing a clear
> question what we are trying
> >> to answer and how the data you want to collect is actually answering
> that question.
> >>
> >> Cheers,
> >>
> >> Uli
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> >> For additional commands, e-mail: dev-help@community.apache.org
> >>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
> For additional commands, e-mail: dev-help@community.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message