hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7204) Localizer errors on archive without any files
Date Fri, 15 Sep 2017 14:55:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Graves updated YARN-7204:
--------------------------------
    Description: 
If a user sends an archive without any files in it (only directories), yarn fails to localize
it with the error below.  I ran into this specifically running spark job but looks generic
to localizer.


 Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_000003
exited with exitCode: -1000
Failing this attempt.Diagnostics: No such file or directory
ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767)
at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218)
at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264)
at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009)
at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012)
at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421)
at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945)
at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
For more detailed output, check the application tracking page: https://rm.com:50708/applicationhistory/app/application_1505252418630_25423
Then click on links to logs of each attempt.
. Failing the application. 

  was:
If a user sends an archive without any files in it (only directories), yarn fails to localize
it with the error below.  I ran into this specifically running spark job but looks generic
to localizer.


 Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_000003
exited with exitCode: -1000
Failing this attempt.Diagnostics: No such file or directory
ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767)
at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218)
at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264)
at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009)
at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012)
at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421)
at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945)
at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
For more detailed output, check the application tracking page: https://rm.com:50508/applicationhistory/app/application_1505252418630_25423
Then click on links to logs of each attempt.
. Failing the application. 


> Localizer errors on archive without any files
> ---------------------------------------------
>
>                 Key: YARN-7204
>                 URL: https://issues.apache.org/jira/browse/YARN-7204
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.8.1
>            Reporter: Thomas Graves
>
> If a user sends an archive without any files in it (only directories), yarn fails to
localize it with the error below.  I ran into this specifically running spark job but looks
generic to localizer.
>  Application application_1505252418630_25423 failed 3 times due to AM Container for appattempt_1505252418630_25423_000003
exited with exitCode: -1000
> Failing this attempt.Diagnostics: No such file or directory
> ENOENT: No such file or directory
> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:767)
> at org.apache.hadoop.fs.DelegateToFileSystem.setPermission(DelegateToFileSystem.java:218)
> at org.apache.hadoop.fs.FilterFs.setPermission(FilterFs.java:264)
> at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1009)
> at org.apache.hadoop.fs.FileContext$11.next(FileContext.java:1005)
> at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
> at org.apache.hadoop.fs.FileContext.setPermission(FileContext.java:1012)
> at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:421)
> at org.apache.hadoop.yarn.util.FSDownload$3.run(FSDownload.java:419)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1945)
> at org.apache.hadoop.yarn.util.FSDownload.changePermissions(FSDownload.java:419)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:365)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:233)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:226)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:214)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> For more detailed output, check the application tracking page: https://rm.com:50708/applicationhistory/app/application_1505252418630_25423
Then click on links to logs of each attempt.
> . Failing the application. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message