community-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kathey Marsden <>
Subject Re: Reviewing the evaluation process
Date Thu, 22 Apr 2010 15:49:44 GMT
On 4/22/2010 7:24 AM, Ross Gardler wrote:
> I'd like to make a few comments about issues that have arisen during 
> the evaluation process for GSoC. I'm going to give my opinion on each, 
> please treat this as lazy consensus - do speak up if you wish to 
> disagree or add more items:
Thank's Ross,
A few comments inline.

> Not enough visibility of the process
> ====================================
> There will always be someone who doesn't read the stuff we send out. 
> Where this is the case I don't think we should worry ourselves.
> Sending to PMCs (including the incubator PMC) is sufficient to reach 
> people. We don't want to send out to committers@ as the project as a 
> whole needs to be behind taking on a GSoC student.
> Reaching PPMCs is more problematic, I think we should continue to rely 
> on incubator mentors taking the message to their projects if they feel 
> it is appropriate.
> --------
> Make it explicit that incubator mentors should pass the message on to 
> PPMCs if appropriate.
> Marking experience mentors up
> =============================
> I really don't like the idea up to 2 points for having been a 
> successful mentor before, firstly it is error prone (e.g. both 
> Bertrand and Luciano have been mentors *and* admins, yet the admin 
> this year was unaware of that). Secondly, just because someone has 
> mentored a student in the past doesn't mean they will be better than 
> another mentor. Finally, mentoring a failing student is, in many ways, 
> more educational than mentoring a successful one.
> We already have "Does the mentor show an understanding of how to 
> mentor a student? (0-4 points)" - I'm more interested in whether the 
> mentor knows what is expected. However, applying this score is difficult.
I think we should also have part of the process where the mentor puts in 
a zero point comment with their evidence that they do understand how to 
mentor a student.  It will give the admins something to reference as 
they probably have no personal understanding of the the mentor's 
understanding or experience.
> --------
> Remove "Has the mentor had a successful student in the past (0-2 
> points)" from the admin rankings
> Add some docs to the ranking process about what admins are looking for 
> with respect to "oes the mentor show an understanding of how to mentor 
> a student? (0-4 points)" (Noirins mail with the subject Admin 
> coordination to alexei.fedotov on code-awards wouild be a good 
> starting point)
+1 Two points is a lot and might discourage new mentors to engage in the 

> Original Ideas are good
> =======================
> Past experience has shown that if a student proposes their own idea 
> and it is accepted the student is going to be strong.
> --------
> Add the following to the mentor ranking:
> Is the project definition and idea originally the mentee's, the
> mentor's or a collaborative effort? (0-2 points, 2 if mentee's idea, 1
> if collaborative, 0 if mentor's)
I think this one is tricky.  In a standards based product like Derby 
there is not that much room for creativity in the initial  "idea" and 
projects  are mostly based on existing Jira entries.  Also really 
everything is collaborative in the community and that should be 
encouraged.    I think I'd rather see this one left out.

Additional items -
1) I think 2 points for a live (non-email) interview is too much.  
Opposing timezones can make this hard to coordinate and I think skills 
in written communication is what is key to success.   If we have to keep 
it, make it clear that IRC is also ok for full points.

2) I think uneven ranking is still something of an issue, but I don't 
know how to address it without putting a lot more work on the admins to 
review the mentor ranking.  I think Alexei's 0 point comments for 
suggested adjustments or more information from students were good, but 
if we all did that, it would mean a lot of comments for the admins to 
review and negotiation that would take more time than we have.

And just a couple doc issues ...

3) Since the ranking process is public (and I am glad that it is), we 
should advise students and mentors to look at it carefully and make sure 
their proposal meets the criteria to the best of their ability.

4) Mentors should be encouraged to  monitor their student's application 
during the critical ranking period to spot any mistakes or injustice and 
speak up at the time.


View raw message