hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kapoor <kap...@capecode.in>
Subject Re: Error in running DistributedShell
Date Thu, 20 Mar 2014 12:35:36 GMT
Is it because I am not running datanode therefore ResourceLocalizationService
is unable to download the jar ?


On 20 March 2014 17:42, Kapoor <kapoor@capecode.in> wrote:

> Hi,
>
> Got code latest code and build it.
>
> Below is the case.
>
> Setup of two machines each having same user and group used for YARN setup.
>
> Only YARN is running that is Resourcemanager and nodemanager
>
> when I ran DistributedShell example I get the below errors and application
> fails
>
> 2014-03-20 17:35:58,776 INFO SecurityLogger.org.apache.hadoop.ipc.Server:
> Auth successful for appattempt_1395316519303_0002_000001 (auth:SIMPLE)
> 2014-03-20 17:35:58,798 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
> Start request for container_1395316519303_0002_01_000001 by user abhishek
> 2014-03-20 17:35:58,798 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
> Creating a new application reference for app application_1395316519303_0002
> 2014-03-20 17:35:58,798 INFO
> org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=abhishek
> IP=192.168.213.80    OPERATION=Start Container Request
> TARGET=ContainerManageImpl    RESULT=SUCCESS
> APPID=application_1395316519303_0002
> CONTAINERID=container_1395316519303_0002_01_000001
> 2014-03-20 17:35:58,798 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Application application_1395316519303_0002 transitioned from NEW to INITING
> 2014-03-20 17:35:58,799 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Adding container_1395316519303_0002_01_000001 to application
> application_1395316519303_0002
> 2014-03-20 17:35:58,799 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application:
> Application application_1395316519303_0002 transitioned from INITING to
> RUNNING
> 2014-03-20 17:35:58,800 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> Container container_1395316519303_0002_01_000001 transitioned from NEW to
> LOCALIZING
> 2014-03-20 17:35:58,800 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got
> event CONTAINER_INIT for appId application_1395316519303_0002
> 2014-03-20 17:35:58,800 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource:
> Resource file:/home/abhishek/DistributedShell/2/AppMaster.jar transitioned
> from INIT to DOWNLOADING
> 2014-03-20 17:35:58,800 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource:
> Resource file:/home/abhishek/DistributedShell/2/shellCommands transitioned
> from INIT to DOWNLOADING
> 2014-03-20 17:35:58,800 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Created localizer for container_1395316519303_0002_01_000001
> 2014-03-20 17:35:58,806 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> Writing credentials to the nmPrivate file
> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/nmPrivate/container_1395316519303_0002_01_000001.tokens.
> Credentials list:
> 2014-03-20 17:35:58,810 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
> Initializing user abhishek
> 2014-03-20 17:35:58,818 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Copying
> from
> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/nmPrivate/container_1395316519303_0002_01_000001.tokens
> to
> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002/container_1395316519303_0002_01_000001.tokens
> 2014-03-20 17:35:58,818 INFO
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: CWD set
> to
> /home/abhishek/hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002
> = file:/home/abhishek
>
> */hadoop2/hadoopdata/data/nodemanagerData/usercache/abhishek/appcache/application_1395316519303_0002
> 2014-03-20 17:35:58,842 WARN
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:abhishek (auth:SIMPLE) cause:java.io.FileNotFoundException: File
> file:/home/abhishek/DistributedShell/2/AppMaster.jar does not exist
> 2014-03-20 17:35:58,843 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService:
> DEBUG: FAILED { file:/home/abhishek/DistributedShell/2/AppMaster.jar,
> 1395316985000, FILE, null }, File
> file:/home/abhishek/DistributedShell/2/AppMaster.jar does not exist*
> 2014-03-20 17:35:58,844 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource:
> Resource file:/home/abhishek/DistributedShell/2/AppMaster.jar transitioned
> from* DOWNLOADING to FAILED*
> 2014-03-20 17:35:58,844 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
> Container container_1395316519303_0002_01_000001 transitioned from
> LOCALIZING to LOCALIZATION_FAILED
> 2014-03-20 17:35:58,844 INFO
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalResourcesTrackerImpl:
> Container container_1395316519303_0002_01_000001 sent RELEASE event on a
> resource request { file:/home/abhishek/DistributedShell/2/AppMaster.jar,
> 1395316985000, FILE, null } not present in cache.
>
>
> Looks like ResourceManager machine extracts DistributedShell jar and
> create home/abhishek/DistributedShell/ locally
> But Node manager machine does not get/create the folder structure and
> download the jar
> * . *
>
> Any pointer to help will be great.
>
>
>
>
> --
> *Thanks and Regards*
> *Abhishek Kapoor*
>
>  *LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/
> <http://in.linkedin.com/in/abhishekkapoorbigdata/>*
> *Twitter: @kapoorSunny*
>



-- 
*Thanks and Regards*
*Abhishek Kapoor*

*LinkedIn : in.linkedin.com/in/abhishekkapoorbigdata/
<http://in.linkedin.com/in/abhishekkapoorbigdata/>*
*Twitter: @kapoorSunny*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message