hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2789) [MR:279] Update the scheduling info on CLI
Date Tue, 11 Oct 2011 23:55:11 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125489#comment-13125489

Eric Payne commented on MAPREDUCE-2789:

Hi Arun. Thanks for catching these issues.

> 1. Need a null check for 'currentAttempt' in RMAppImpl.

> 2. In RMAppAttemptImpl.getApplicationResourceUsageReport: 
>   1. You need to acquire the readLock.
I don't think the read lock would be effective or necessary here. Please tell me if my logic
is faulty :D  :
* Within {{RMAppAttemptImpl.java}}, {{getApplicationResourceUsageReport()}} calls {{scheduler.getSchedulerAppInfo(...)}}
to get an instance of the SchedulerAppReport object. This instance variable is local to {{getApplicationResourceUsageReport()}}
and contains copies of the {{live}} node and {{reserved}} node lists, not pointers to the
actual lists themselves.
** Even if this instance of SchedulerAppReport contained pointers to the {{live}} and {{reserved}}
node lists, the private readlock within the RMAppAttemptImpl class would not be honored by
anything changing those lists, so this readlock would not be effective.
** Since this instance of SchedulerAppReport and its container {{live}} and {{reserved}} variables
are local to {{getApplicationResourceUsageReport()}}, I don't think one thread can access
the local variables of another thread, even if they are running in the same code. So, I don't
think the readlock is needed, even for the sake of re-entrance.

>   2. Reserved containers loop needs a +=, not =
Done. Good catch.

> 3. In ApplicationResourceUsageReport, please use Resource rather than int for all parameters/return-values.
We don't want to tie it to just memory.
I did this for the ones that are tracking resources. The others are simply countin containers.

> 4. JobStatus 
>   1. Minor nit - please use 'Reserved' rather than 'Rsvd' for apis, we try to avoid these
in api names (maybe use expansion for variable names too?)

>   2. I don't get why we need (get,set)ApplicationMasterInfo - we already have (get,set)TrackingUrl?
I removed the JobStatus.(set,get)ApplicationMasterInfo() APIs and just used the (set,get)SchedulingInfo

> [MR:279] Update the scheduling info on CLI
> ------------------------------------------
>                 Key: MAPREDUCE-2789
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2789
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ramya Sunil
>            Assignee: Eric Payne
>             Fix For: 0.23.0, 0.24.0
>         Attachments: MAPREDUCE-2789.v1.txt, MAPREDUCE-2789.v2.txt, MAPREDUCE-2789.v3.txt
> The scheduling information such as number of containers running, memory usage and reservations
per job is not available on bin/mapred job -list CLI.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message