hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bertrand Dechoux (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3562) Concurrency issues in MultipleOutputs,JobControl,Counters
Date Tue, 03 Jul 2012 09:24:10 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405718#comment-13405718
] 

Bertrand Dechoux commented on MAPREDUCE-3562:
---------------------------------------------

About point 2, I would like to express another concurrency issue. I simply would like to print
out the state of my JobControl ie its status but also the size of all jobs lists (and later
the name of the jobs inside). My issue is that there is currently no way to have such consistent
information from outside the job context (ie another thread).

Eg I call getRunningJobList() and then getSuccessfulJobList() : my job which was running before
the first call, may be successful before the second call...

I assume that the user is responsible for running the JobControl in a separate Thread and
stopping it when needed. I don't see any clean way to extend the current JobControl to have
that consistent view.

About the pach, we agree adding synchronized would not help.
                
> Concurrency issues in MultipleOutputs,JobControl,Counters
> ---------------------------------------------------------
>
>                 Key: MAPREDUCE-3562
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3562
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ravi Teja Ch N V
>            Assignee: Ravi Teja Ch N V
>         Attachments: MAPREDUCE-3562.patch
>
>
> bq.MultipleOutputs 
>   The close of recordwriters should be synchronized. 
>   public void close() throws IOException, InterruptedException { 
>     for (RecordWriter writer : recordWriters.values()) { 
>       writer.close(context); 
> bq.JobControl.java 
>   the getters of the jobs to be synchronized. 
> bq.Counters.java 
>    makeEscapedCompactString to be made synchronized. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message