community-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rich Bowen <>
Subject Re: Does GSoC help develop communities?
Date Mon, 05 Dec 2016 21:24:18 GMT
So ... stepping back a bit here. Are you saying that even attempting to
measure outcomes is harmful, because we might draw the wrong conclusions?

This is a completely new notion to me.

You say:

"I am reluctant to simply collect some data because I am missing a clear
question what we are trying to answer and how the data you want to
collect is actually answering that question."

I do have a clear question. My question is whether participating in the
GSoC contributes to our goal of Community Development. Does it develop
our community in any measurable way? I assume, at this point, that it
does, or we wouldn't keep doing it. In what way has it had measurable
impact on our community?

Things that we can clearly measure is adding participants in our project
communities, and artifacts added to our projects' revision control
repositories. Measuring other things is more difficult, and can be taken
as a later task. Easy things first.

The larger question of how and what we measure in the Community
Development PMC as a whole hinges on this, too. I'm curious if you're
resistant to that, too?


On 12/05/2016 04:00 PM, Ulrich Stärk wrote:
> On 05.12.16 17:54, Rich Bowen wrote:
>> On 12/05/2016 07:41 AM, Ulrich Stärk wrote:
>>>> Or, at the very least, can we make a commitment to track this data going
>>>>> forward?
>>> Let me play the devil's advocate here: What for?
>>> GSoC is completely free for the ASF (on the contrary, we even get a small amount
for every accepted
>>> student that we can than put towards fulfilling our goals) and as long as we
have volunteers willing
>>> to organize it and mentor students we can assume that at least those volunteers
are seeing value in
>>> it. Why the stats other than for satisfying our curiosity?
>> Or, perhaps, let me give a different answer.
>> I participated in GSoC as a mentor for $WorkProject. While it didn't
>> "cost" me anything in dollars, it cost me probably 200 hours of my time.
>> I know that other projects at work put more time in, and some less.
>> This is an enormous cost to me, as an employee. So it behooves me to
>> measure the benefit to the project. A student received payment to write
>> code that was discarded. And I (and several of my colleagues) spent a
>> huge amount of time, which I could have spent on other things, mentoring
>> that student. Benefit to project, pretty heavily negative.
> In that case you should have failed the student pretty early as I tell mentors every
year and as
> documented in the mentor guide.
>> So, now, here we are at the ASF, doing GSoC with our projects, and
>> promoting it to them as a benefit. Does it actually benefit them, or is
>> it merely siphoning off time that could be spent on other things.
> If a mentor feels it is the latter, they should immediately fail the student.
>> To folks that say we can't measure that, I strongly disagree. There are
>> two measures that are obvious and easy.
>> 1) What % of GSoC student are still active on the project 6 months, 12
>> months, 18 months after the project. (We can debate the definition of
>> "active" all you like.
> This implies that the program is only successful if a certain number of students sticks
around. And
> this is exactly what I'm arguing against. IMO the program is successful as soon as a
student has
> some exposure to one of our communities.
>> 2) What % of code developed by GSoC students actually becomes a part of
>> the project codebase at the end of the project?
> This has the same problems as trying to measure productivity from code. What is the percentage
> telling us about the question we want to answer? Also see above: If the code cannot be
part of the
> project codebase the student should be failed.
>> I would maintain that #1 is part of our charter as ComDev, and #2 is
>> part of what projects should be made aware of before they sign on.
>> Again, I'm really not asking for a lot of data here. But I do think that
>> it's part of a responsible accounting for participating in GSoC. If, for
>> example, it is actually hindering projects, don't we want to know that.
>> I did *not* participate in GSoC again, on $WorkProject, because it
>> clearly hindered my project.
> I am reluctant to simply collect some data because I am missing a clear question what
we are trying
> to answer and how the data you want to collect is actually answering that question.
> Cheers,
> Uli
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

Rich Bowen - - @rbowen - @apachecon

View raw message