hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-927) Cleanup of task-logs should happen in TaskTracker instead of the Child
Date Tue, 09 Mar 2010 04:53:29 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842907#action_12842907
] 

Amareshwari Sriramadasu commented on MAPREDUCE-927:
---------------------------------------------------

Test failure TestBadRecords is not related to the patch. Test failed because of many ZipException
and NoClassDefFoundError.
log for the exception:
{noformat}
2010-03-08 11:30:53,708 WARN  mapred.TaskTracker (TaskTracker.java:startNewTask(2223)) - Error
initializing attempt_20100308113025258_0001_m_000000_1:
java.lang.RuntimeException: java.util.zip.ZipException: error reading zip file
	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1715)
	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1529)
	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1475)
	at org.apache.hadoop.conf.Configuration.get(Configuration.java:564)
	at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:1892)
	at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:397)
	at org.apache.hadoop.mapred.TaskTracker.localizeJobFiles(TaskTracker.java:1035)
	at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:961)
	at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2218)
	at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2183)
Caused by: java.util.zip.ZipException: error reading zip file
	at java.util.zip.ZipFile.read(Native Method)
	at java.util.zip.ZipFile.access$1200(ZipFile.java:29)
	at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:447)
	at java.util.zip.ZipFile$1.fill(ZipFile.java:230)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:141)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:105)
	at java.io.FilterInputStream.read(FilterInputStream.java:66)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2910)
	at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:704)
	at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:225)
	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
	at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1628)
	... 9 more
{noformat}
The test passed on my machine.

> Cleanup of task-logs should happen in TaskTracker instead of the Child
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-927
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-927
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: security, tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Vinod K V
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.22.0
>
>         Attachments: patch-927-1.txt, patch-927-2.txt, patch-927-3.txt, patch-927.txt
>
>
> Task logs' cleanup is being done in Child now. This is undesirable atleast for two reasons:
1) failures while cleaning up will affect the user's tasks, and 2) the task's wall time will
get affected due to operations that TT actually should own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message