hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From daemeon reiydelle <daeme...@gmail.com>
Subject Re: Localizer failed but it works after I restart yarn
Date Fri, 06 Feb 2015 05:16:53 GMT
My first concern is that temp is a virtual file system and cannot exceed
real memory plus swap space. You may not see file system full errors flow
up as the root cause of the stack trace, but you will see null pointers. So
move your temp directory onto your file system (~/Documents/HDFSTMP or
something) and retry.



*.......*






*“Life should not be a journey to the grave with the intention of arriving
safely in apretty and well preserved body, but rather to skid in broadside
in a cloud of smoke,thoroughly used up, totally worn out, and loudly
proclaiming “Wow! What a Ride!” - Hunter ThompsonDaemeon C.M. ReiydelleUSA
(+1) 415.501.0198London (+44) (0) 20 8144 9872*

On Thu, Feb 5, 2015 at 7:36 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> BTW, I am using hadoop 2.6 and it's a single node cluster on mac.
>
>
>
> On Fri, Feb 6, 2015 at 11:35 AM, Jeff Zhang <zjffdu@gmail.com> wrote:
>
>> I intermittently meet this issue of Localizer failed, but after I restart
>> the yarn, then the problem is gone. Is this a known issue ?
>>
>> Here's log in node manager
>>
>>
>> 2015-02-06 11:31:24,898 INFO
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
>> Localizer CWD set to
>> /tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> =
>> file:/tmp/hadoop-jzhang/nm-local-dir/usercache/jzhang/appcache/application_1423126607079_0007
>> 2015-02-06 11:31:24,906 INFO
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
>> Localizer failed
>> java.lang.NullPointerException
>>     at java.util.zip.ZipFile.getZipEntry(ZipFile.java:531)
>>     at java.util.zip.ZipFile.access$900(ZipFile.java:56)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:513)
>>     at java.util.zip.ZipFile$1.nextElement(ZipFile.java:483)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:244)
>>     at java.util.jar.JarFile$1.nextElement(JarFile.java:239)
>>     at sun.misc.URLClassPath$JarLoader.validIndex(URLClassPath.java:803)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:937)
>>     at sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:848)
>>     at sun.misc.URLClassPath$JarLoader.findResource(URLClassPath.java:818)
>>     at sun.misc.URLClassPath.findResource(URLClassPath.java:176)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:551)
>>     at java.net.URLClassLoader$2.run(URLClassLoader.java:549)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findResource(URLClassLoader.java:548)
>>     at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
>>     at
>> java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
>>     at org.apache.xerces.parsers.SecuritySupport$6.run(Unknown Source)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at
>> org.apache.xerces.parsers.SecuritySupport.getResourceAsStream(Unknown
>> Source)
>>     at
>> org.apache.xerces.parsers.ObjectFactory.findJarServiceProvider(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.ObjectFactory.createObject(Unknown
>> Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.parsers.DOMParser.<init>(Unknown Source)
>>     at org.apache.xerces.jaxp.DocumentBuilderImpl.<init>(Unknown Source)
>>     at
>> org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(Unknown
>> Source)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2402)
>>     at
>> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2364)
>>     at
>> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2281)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1019)
>>     at org.apache.hadoop.conf.Configuration.set(Configuration.java:991)
>>     at
>> org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1791)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:407)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:129)
>>     at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1088)
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Mime
View raw message