hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White" <tom.e.wh...@gmail.com>
Subject Re: Removing JMM issues
Date Mon, 17 Nov 2008 14:09:29 GMT
This would be an interesting project and useful to have. I would
suggest starting by running a static analysis tool like Chord
(http://code.google.com/p/jchord/) and creating a series of Jira
issues to fix problems. Later on, we could integrate it into the
Hudson patch checking process, just like FindBugs is now. Creating a
wiki page describing coding best practices for concurrency might be
useful too.

Cheers,
Tom

On Mon, Nov 17, 2008 at 12:09 PM, Peter Veentjer <alarmnummer@gmail.com> wrote:
> Hi Ted,
>
> one of the easy to find problems is spinning on a non volatile
> variable without changing the value in the loop and without additional
> synchronization (at least I didn't find it in a few seconds).
>
> examples:
> AgentControllerSocketListener.closing
> HttpConnector.stopMe
> IndexUpdateReducer.closed
> SecundaryNodeName.shouldRun
> voTask.hasNext
>
> These can all be fixed by making the field volatile.
>
> These are the easy ones that can be found with static analysis tools,
> but I bet there are a lot of more harder to find ones.
>
> I think the Hadoop project would benefit from a structural approach to
> solving these problems instead of just fixing these bugs. That is what
> I want to help with but I can't do it without support of the
> leading-developers of the Hadoop community.
>
> One of the things we need to agree upon is for example:
> making fields that only are set in the constructor, final. This makes
> analysis a lot easier.
>
> On Sun, Nov 16, 2008 at 11:37 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
>>
>> Can you give some examples?
>>
>> Sent from my iPhone
>>
>> On Nov 16, 2008, at 13:09, "Peter Veentjer" <alarmnummer@gmail.com> wrote:
>>
>>> Hi Guys,
>>>
>>> I have had a quick look at the source code of Hadoop and it appears
>>> there there are some issues with the JMM. In some places it is done
>>> correctly,  in some places partially and in some places it incorrect.
>>> There also are some design issues with concurrency as well and I think
>>> the Hadoop project could benefit from overall solution instead of just
>>> putting out small fires. So who are the guys to get in touch with?
>>> Together with the Hadoop developers I want to further improve the
>>> quality of this very interesting project.
>>>
>>> Some information about me:
>>>
>>> my blog
>>> http://pveentjer.wordpress.com
>>>
>>> and some of my open source projects:
>>> http://prometheus.codehaus.org
>>> http://multiverse.googlecode.com
>>> http://concurrency-detector.googlecode.com
>>
>

Mime
View raw message