flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Flink batch runs OK but Yarn container fails in batch mode with -m yarn-cluster
Date Thu, 03 Sep 2015 08:48:29 GMT
Hi Arnaud,

I think that's a bug ;)
I'll file a JIRA to fix it for the next release.

On Thu, Sep 3, 2015 at 10:26 AM, LINZ, Arnaud <ALINZ@bouyguestelecom.fr>
wrote:

> Hi,
>
>
>
> I am wondering why, despite the fact that my java main() methods runs OK
> and exit with 0 code value, the Yarn container status set by the englobing
> flink execution is FAILED with diagnostic "Flink YARN Client requested
> shutdown."?
>
>
>
> Command line :
>
> flink run -m yarn-cluster -yn 20 -ytm 8192 -yqu batch1 -ys 8 --class
> <myMain> <myJar> <myParams>
>
>
>
> End of yarn log :
>
>
>
> Status of job 6ac47ddc8331ffd0b1fa9a3b5a551f86 (KUBERA-GEO-BRUT2SEGMENT)
> changed to FINISHED.
>
> 10:03:00,618 INFO
> org.apache.flink.yarn.ApplicationMaster$$anonfun$2$$anon$1    - Stopping
> YARN JobManager with status FAILED and diagnostic Flink YARN Client
> requested shutdown.
>
> 10:03:00,625 INFO
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Waiting for
> application to be successfully unregistered.
>
> 10:03:00,874 INFO
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> Closing proxy : h1r2dn12.bpa.bouyguestelecom.fr:45454
>
> (… more closing proxy …)
>
> 10:03:00,877 INFO
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> Closing proxy : h1r2dn01.bpa.bouyguestelecom.fr:45454
>
> 10:03:00,883 INFO
> org.apache.flink.yarn.ApplicationMaster$$anonfun$2$$anon$1    - Stopping
> JobManager akka://flink/user/jobmanager#1737010364.
>
> 10:03:00,895 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator         - Shutting
> down remote daemon.
>
> 10:03:00,896 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator         - Remote
> daemon shut down; proceeding with flushing remote transports.
>
> 10:03:00,918 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator         - Remoting
> shut down.
>
>
>
> End of log4j log:
>
>
>
> 2015:09:03 10:03:00 (main) - INFO -
> com.bouygtel.kuberasdk.main.Application.mainMethod - Fin ok traitement
>
> 2015:09:03 10:03:00 (Thread-14) - INFO - Classe Inconnue.Methode Inconnue
> - Shutting down FlinkYarnCluster from the client shutdown hook
>
> 2015:09:03 10:03:00 (Thread-14) - INFO - Classe Inconnue.Methode Inconnue
> - Sending shutdown request to the Application Master
>
> 2015:09:03 10:03:00 (flink-akka.actor.default-dispatcher-2) - INFO -
> Classe Inconnue.Methode Inconnue - Sending StopYarnSession request to
> ApplicationMaster.
>
> 2015:09:03 10:03:00 (flink-akka.actor.default-dispatcher-2) - INFO -
> Classe Inconnue.Methode Inconnue - Remote JobManager has been stopped
> successfully. Stopping local application client
>
> 2015:09:03 10:03:00 (flink-akka.actor.default-dispatcher-2) - INFO -
> Classe Inconnue.Methode Inconnue - Stopped Application client.
>
> 2015:09:03 10:03:00 (flink-akka.actor.default-dispatcher-15) - INFO -
> Classe Inconnue.Methode Inconnue - Shutting down remote daemon.
>
> 2015:09:03 10:03:00 (flink-akka.actor.default-dispatcher-15) - INFO -
> Classe Inconnue.Methode Inconnue - Remote daemon shut down; proceeding with
> flushing remote transports.
>
> 2015:09:03 10:03:00 (flink-akka.actor.default-dispatcher-15) - INFO -
> Classe Inconnue.Methode Inconnue - Remoting shut down.
>
> 2015:09:03 10:03:00 (Thread-14) - INFO - Classe Inconnue.Methode Inconnue
> - Deleting files in hdfs://
> h1r1nn01.bpa.bouyguestelecom.fr:8020/user/datcrypt/.flink/application_1441011714087_0730
>
> 2015:09:03 10:03:00 (Thread-15) - INFO - Classe Inconnue.Methode Inconnue
> - Application application_1441011714087_0730 finished with state FINISHED
> and final state FAILED at 1441267380623
>
> 2015:09:03 10:03:00 (Thread-14) - WARN - Classe Inconnue.Methode Inconnue
> - The short-circuit local reads feature cannot be used because libhadoop
> cannot be loaded.
>
> 2015:09:03 10:03:01 (Thread-14) - INFO - Classe Inconnue.Methode Inconnue
> - YARN Client is shutting down
>
>
>
> Greetings,
>
> Arnaud
>
> ------------------------------
>
> L'intégrité de ce message n'étant pas assurée sur internet, la société
> expéditrice ne peut être tenue responsable de son contenu ni de ses pièces
> jointes. Toute utilisation ou diffusion non autorisée est interdite. Si
> vous n'êtes pas destinataire de ce message, merci de le détruire et
> d'avertir l'expéditeur.
>
> The integrity of this message cannot be guaranteed on the Internet. The
> company that sent this message cannot therefore be held liable for its
> content nor attachments. Any unauthorized use or dissemination is
> prohibited. If you are not the intended recipient of this message, then
> please delete it and notify the sender.
>

Mime
View raw message