community-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ulrich Stärk <...@spielviel.de>
Subject Re: Does GSoC help develop communities?
Date Tue, 06 Dec 2016 08:36:13 GMT
On 05.12.16 22:24, Rich Bowen wrote:
> So ... stepping back a bit here. Are you saying that even attempting to
> measure outcomes is harmful, because we might draw the wrong conclusions?
> 
> This is a completely new notion to me.
> 
> You say:
> 
> "I am reluctant to simply collect some data because I am missing a clear
> question what we are trying to answer and how the data you want to
> collect is actually answering that question."
> 
> I do have a clear question. My question is whether participating in the
> GSoC contributes to our goal of Community Development. Does it develop
> our community in any measurable way? I assume, at this point, that it
> does, or we wouldn't keep doing it. In what way has it had measurable
> impact on our community?
> 
> Things that we can clearly measure is adding participants in our project
> communities, and artifacts added to our projects' revision control
> repositories. Measuring other things is more difficult, and can be taken
> as a later task. Easy things first.

And here is what I'm afraid of: By focusing on those two simple dimensions only we will from
my
experience end up with a pretty skewed result. The dimensions you propose don't cover code
living on
outside our repositories as add-on modules, they don't cover indirect contributions, etc.

> 
> The larger question of how and what we measure in the Community
> Development PMC as a whole hinges on this, too. I'm curious if you're
> resistant to that, too?

I am resistant to what you are proposing to measure. I am resistant to simplifying metrics.
Those
tend to do more harm than good.

Uli

> 
> --Rich
> 
> 
> 
> On 12/05/2016 04:00 PM, Ulrich Stärk wrote:
>> On 05.12.16 17:54, Rich Bowen wrote:
>>>
>>>
>>> On 12/05/2016 07:41 AM, Ulrich Stärk wrote:
>>>>> Or, at the very least, can we make a commitment to track this data going
>>>>>> forward?
>>>> Let me play the devil's advocate here: What for?
>>>>
>>>> GSoC is completely free for the ASF (on the contrary, we even get a small
amount for every accepted
>>>> student that we can than put towards fulfilling our goals) and as long as
we have volunteers willing
>>>> to organize it and mentor students we can assume that at least those volunteers
are seeing value in
>>>> it. Why the stats other than for satisfying our curiosity?
>>>
>>> Or, perhaps, let me give a different answer.
>>>
>>> I participated in GSoC as a mentor for $WorkProject. While it didn't
>>> "cost" me anything in dollars, it cost me probably 200 hours of my time.
>>> I know that other projects at work put more time in, and some less.
>>>
>>> This is an enormous cost to me, as an employee. So it behooves me to
>>> measure the benefit to the project. A student received payment to write
>>> code that was discarded. And I (and several of my colleagues) spent a
>>> huge amount of time, which I could have spent on other things, mentoring
>>> that student. Benefit to project, pretty heavily negative.
>>
>> In that case you should have failed the student pretty early as I tell mentors every
year and as
>> documented in the mentor guide.
>>
>>>
>>> So, now, here we are at the ASF, doing GSoC with our projects, and
>>> promoting it to them as a benefit. Does it actually benefit them, or is
>>> it merely siphoning off time that could be spent on other things.
>>
>> If a mentor feels it is the latter, they should immediately fail the student.
>>
>>>
>>> To folks that say we can't measure that, I strongly disagree. There are
>>> two measures that are obvious and easy.
>>>
>>> 1) What % of GSoC student are still active on the project 6 months, 12
>>> months, 18 months after the project. (We can debate the definition of
>>> "active" all you like.
>>
>> This implies that the program is only successful if a certain number of students
sticks around. And
>> this is exactly what I'm arguing against. IMO the program is successful as soon as
a student has
>> some exposure to one of our communities.
>>
>>>
>>> 2) What % of code developed by GSoC students actually becomes a part of
>>> the project codebase at the end of the project?
>>
>> This has the same problems as trying to measure productivity from code. What is the
percentage
>> telling us about the question we want to answer? Also see above: If the code cannot
be part of the
>> project codebase the student should be failed.
>>
>>>
>>> I would maintain that #1 is part of our charter as ComDev, and #2 is
>>> part of what projects should be made aware of before they sign on.
>>>
>>> Again, I'm really not asking for a lot of data here. But I do think that
>>> it's part of a responsible accounting for participating in GSoC. If, for
>>> example, it is actually hindering projects, don't we want to know that.
>>>
>>> I did *not* participate in GSoC again, on $WorkProject, because it
>>> clearly hindered my project.
>>>
>>>
>>
>> I am reluctant to simply collect some data because I am missing a clear question
what we are trying
>> to answer and how the data you want to collect is actually answering that question.
>>
>> Cheers,
>>
>> Uli
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
>> For additional commands, e-mail: dev-help@community.apache.org
>>
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org


Mime
View raw message