hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2193) 13 Findbugs warnings on trunk and branch-0.22
Date Thu, 17 Feb 2011 11:59:24 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995776#comment-12995776
] 

Chris Douglas commented on MAPREDUCE-2193:
------------------------------------------

bq. I'd like someone else to take a look at adding 'synchronized' to getTasks() in addition
to me - it's just scary enough that I think an extra reviewer is prudent.

I tried to find some history: http://s.apache.org/T99 (from MAPREDUCE-1316). The synchronization
was intentionally left out, but only to be consistent with the existing code.

The synchronization around JIP is difficult to track. Its omission here appears to admit the
possibility that a query of an initializing job could iterate over a partially constructed
set of TIPs and cause NPEs (e.g. via the JSP), but I couldn't find a place where this is a
serious problem. After the job is initialized, none of the references to member fields become
stale. On the other hand, I looked through all the callers of getTasks(), and most are holding
the JT lock already (mostly schedulers, during heartbeat processing) so I doubt that synchronizing
on the JIP here would cause problems.

One could fix getTasks() to be threadsafe by changing create{Map,Reduce}Tasks and initSetupCleanupTasks
to assign the initialized array of tasks to the member field. The current behavior constructs
it in-place, which (AFAICT) means that a caller of getTasks() could read partially-constructed
TIPs. It's probably better to make getTasks() synchronized, to avoid having getTasks() return
partially constructed, inconsistent JIP state.

> 13 Findbugs warnings on trunk and branch-0.22
> ---------------------------------------------
>
>                 Key: MAPREDUCE-2193
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2193
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.22.0, 0.23.0
>            Reporter: Nigel Daley
>            Assignee: Harsh J Chouraria
>            Priority: Blocker
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: findbugsWarnings.html, hadoop-findbugs-report.html, hadoop-findbugs-report.html,
mapreduce.trunk.findbugs.r1.diff, mapreduce.trunk.findbugs.r2.diff, mapreduce.trunk.findbugs.r3.diff,
mapreduce.trunk.findbugs.r4.diff
>
>
> There are 13 findbugs warnings on trunk.  See attached html file.  These must be fixed
or filtered out to get back to 0 warnings.  The OK_FINDBUGS_WARNINGS property in src/test/test-patch.properties
should also be set to 0 in the patch that fixes this issue.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message