Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <306099511.1207147884594.JavaMail.jira@brutus>
Date: Wed, 2 Apr 2008 07:51:24 -0700 (PDT)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3135) if the 'mapred.system.dir' in the
 client jobconf is different from the JobTracker's value job submission
 fails
In-Reply-To: <661424110.1206960746126.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584605#action_12584605 ] 

Hadoop QA commented on HADOOP-3135:
-----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12379126/patch-3135.txt
against trunk revision 643282.

    @author +1.  The patch does not contain any @author tags.

    tests included +1.  The patch appears to include 3 new or modified tests.

    javadoc +1.  The javadoc tool did not generate any warning messages.

    javac +1.  The applied patch does not generate any new javac compiler warnings.

    release audit +1.  The applied patch does not generate any new release audit warnings.

    findbugs +1.  The patch does not introduce any new Findbugs warnings.

    core tests -1.  The patch failed core unit tests.

    contrib tests +1.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2123/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2123/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2123/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2123/console

This message is automatically generated.

> if the 'mapred.system.dir' in the client jobconf is different from the JobTracker's value job submission fails
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3135
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3135
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.16.1
>         Environment: all
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Blocker
>             Fix For: 0.17.0
>
>         Attachments: patch-3135.txt
>
>
> Until Hadoop 0.13 or so, at submission time the full path of the job.xml and all supporting files in DFS was given by the client to the jobtracker.
> Since 0.15 onwards (we did not test 0.14) the jobclient is obtaining the job ID from the jobtracker and creating the directory for all the supporting files using the a system-dir computed from the local jobconf.
> Line 696-7 in the JobClient:
>     String jobId = jobSubmitClient.getNewJobId();
>     Path submitJobDir = new Path(job.getSystemDir(), jobId);
> This makes submissions to fail when the value of the 'mapred.system.dir' on the client is different from the one in the JobTracker.
> A simple way o fixing this would be to introduce a new method in the JobSubmissionProtocol 'getSystemDir()' that would return the jobtracker system dir and use that dir for uploading all the files on submission.
> ----
> For the future: A more comprehensive way of this doing would to obtain a base jobConf from the jobtracker, carrying final information for each element and the construct the job.xml on the client using the final semantics. And, in this case the 'mapred.system.dir' property should be set as final in the jobtracker. As there may be some configuration properties that are sensitive and for security reasons should not be exposed to the clients a new flag 'private' could be introduced and only properties that don't have the 'private' flag would be send over from the jobtracker to the jobclient for job.xml resolution.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.