hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sam rash (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5140) setting fs.default.name to an invalid URI format kills init thread in JobTracker
Date Thu, 29 Jan 2009 02:28:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668293#action_12668293
] 

sam rash commented on HADOOP-5140:
----------------------------------

first stack trace:
2009-01-28 18:15:41,531 ERROR org.apache.hadoop.mapred.EagerTaskInitializationListener: Job
initialization failed:
java.lang.IllegalArgumentException
        at java.net.URI.create(URI.java:842)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:128)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:208)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
        at org.apache.hadoop.mapred.JobHistory$JobInfo.getJobHistoryFileName(JobHistory.java:630)
        at org.apache.hadoop.mapred.JobHistory$JobInfo.logSubmitted(JobHistory.java:803)
        at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:353)
        at org.apache.hadoop.mapred.EagerTaskInitializationListener$JobInitThread.run(EagerTaskInitializationListener.java:55)
        at java.lang.Thread.run(Thread.java:637)
Caused by: java.net.URISyntaxException: Illegal character in authority at index 7: hdfs://${name}
        at java.net.URI$Parser.fail(URI.java:2809)
        at java.net.URI$Parser.parseAuthority(URI.java:3147)
        at java.net.URI$Parser.parseHierarchical(URI.java:3058)
        at java.net.URI$Parser.parse(URI.java:3014)
        at java.net.URI.<init>(URI.java:578)
        at java.net.URI.create(URI.java:840)
        ... 8 more

code with the problem:

class JobInitThread implements Runnable {
    public void run() {
      JobInProgress job;
      while (true) {
        job = null;
        try {
          synchronized (jobInitQueue) {
            while (jobInitQueue.isEmpty()) {
              jobInitQueue.wait();
            }
            job = jobInitQueue.remove(0);
          }
          job.initTasks();
        } catch (InterruptedException t) {
          break;
        } catch (Throwable t) {
          LOG.error("Job initialization failed:\n" +
                    StringUtils.stringifyException(t));
          if (job != null) {
            job.fail();
          }
        }
      }
    }
  }

job.fail() eventually calls garbageCollect() which trys to get the filesystem from the jobconf.
 The same exceptoin as above will occur when it trys to cleanup the dfs tmp dir, but this
time it's not caught (being thrown inside the catch() block) and kills the JobInitThread

> setting fs.default.name to an invalid URI format kills init thread in JobTracker
> --------------------------------------------------------------------------------
>
>                 Key: HADOOP-5140
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5140
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: sam rash
>            Priority: Minor
>
> If you set fs.default.name in a JobConf object to something that causes java.net.URI
to throw an IllegalArgumentException, the job not only fails initalization, but kills the
JobInitThread

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message