hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Removing JMM issues
Date Mon, 17 Nov 2008 14:37:03 GMT
Peter Veentjer wrote:
> Hi Ted,
> one of the easy to find problems is spinning on a non volatile
> variable without changing the value in the loop and without additional
> synchronization (at least I didn't find it in a few seconds).
> examples:
> AgentControllerSocketListener.closing
> HttpConnector.stopMe
> IndexUpdateReducer.closed
> SecundaryNodeName.shouldRun
> voTask.hasNext
> These can all be fixed by making the field volatile.
> These are the easy ones that can be found with static analysis tools,
> but I bet there are a lot of more harder to find ones.

One of the problems with concurrency issues is that they are hard to 
test for -it's hard to create tests to show that the problem exists. 
Another is that the main services -namenode, secondary namenode, etc, 
all run (in production) in their own processes, so can get away with 
concurrency risks and static shared code that aren't so appealing in 
shared processes.

> I think the Hadoop project would benefit from a structural approach to
> solving these problems instead of just fixing these bugs. That is what
> I want to help with but I can't do it without support of the
> leading-developers of the Hadoop community.

1. I don't see anyone being against this, though you would have to start 
with education. For example, it took me a bit to work out that you were 
using JMM as an acronym for Java Memory Model.

2. I think we'd need to prioritise where the biggest risks are.

> One of the things we need to agree upon is for example:
> making fields that only are set in the constructor, final. This makes
> analysis a lot easier.

It does, but it also makes subclassing trickier as subclassed instances 
don't get a look in or an opportunity to override the values -even if 
they have methods you can use to evaluate the subclassed values, the 
fact that these are called from the parent's constructor makes them a 
risk all on their own

View raw message