hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Veentjer" <alarmnum...@gmail.com>
Subject Re: Removing JMM issues
Date Mon, 17 Nov 2008 12:09:08 GMT
Hi Ted,

one of the easy to find problems is spinning on a non volatile
variable without changing the value in the loop and without additional
synchronization (at least I didn't find it in a few seconds).

examples:
AgentControllerSocketListener.closing
HttpConnector.stopMe
IndexUpdateReducer.closed
SecundaryNodeName.shouldRun
voTask.hasNext

These can all be fixed by making the field volatile.

These are the easy ones that can be found with static analysis tools,
but I bet there are a lot of more harder to find ones.

I think the Hadoop project would benefit from a structural approach to
solving these problems instead of just fixing these bugs. That is what
I want to help with but I can't do it without support of the
leading-developers of the Hadoop community.

One of the things we need to agree upon is for example:
making fields that only are set in the constructor, final. This makes
analysis a lot easier.

On Sun, Nov 16, 2008 at 11:37 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
>
> Can you give some examples?
>
> Sent from my iPhone
>
> On Nov 16, 2008, at 13:09, "Peter Veentjer" <alarmnummer@gmail.com> wrote:
>
>> Hi Guys,
>>
>> I have had a quick look at the source code of Hadoop and it appears
>> there there are some issues with the JMM. In some places it is done
>> correctly,  in some places partially and in some places it incorrect.
>> There also are some design issues with concurrency as well and I think
>> the Hadoop project could benefit from overall solution instead of just
>> putting out small fires. So who are the guys to get in touch with?
>> Together with the Hadoop developers I want to further improve the
>> quality of this very interesting project.
>>
>> Some information about me:
>>
>> my blog
>> http://pveentjer.wordpress.com
>>
>> and some of my open source projects:
>> http://prometheus.codehaus.org
>> http://multiverse.googlecode.com
>> http://concurrency-detector.googlecode.com
>

Mime
View raw message