Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
Subject: Re: Flink on Yarn - ApplicationMaster command
To: user@flink.apache.org
References: <57162676.4040008@kth.se>
 <CAGco--ZwLsP_adpmt8KOkQ7FDfm1OXDvGU7Y0i3VNyxZN6khSg@mail.gmail.com>
From: Theofilos Kakantousis <tkak@kth.se>
Message-ID: <5716A238.3060309@kth.se>
Date: Wed, 20 Apr 2016 00:25:12 +0300
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101
 Thunderbird/38.7.2
MIME-Version: 1.0
In-Reply-To: 
 <CAGco--ZwLsP_adpmt8KOkQ7FDfm1OXDvGU7Y0i3VNyxZN6khSg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Max,

Thank you for your reply. Exactly, I want to setup the Yarn cluster and 
submit a job through code and not using cmd client.
I had done what you suggested, I used part of the deploy method to write 
my own code that starts up the cluster which seems to be working fine.

Could you point me to some examples how to use the Client you mention?

Cheers,
Theofilos

On 2016-04-19 16:35, Maximilian Michels wrote:
> Hi Theofilos,
>
> I'm not sure whether I understand correctly what you are trying to do.
> I'm assuming you don't want to use the command-line client.
>
> You can setup the Yarn cluster in your code manually using the
> FlinkYarnClient class. The deploy() method will give you a
> FlinkYarnCluster which you can use to connect to the deployed cluster.
> Then get the JobManager address and use the Client class to submit
> Flink jobs to the cluster. I have to warn you that these classes are
> subject to change in Flink 1.1.0 and above.
>
> Let me know if the procedure works for you.
>
> Cheers,
> Max
>
> On Tue, Apr 19, 2016 at 2:37 PM, Theofilos Kakantousis <tkak@kth.se> wrote:
>> Hi everyone,
>>
>> I'm using Flink 0.10.1 and hadoop 2.4.0 to implement a client that submits a
>> flink application to Yarn. To keep it simple I use the ConnectedComponents
>> app from flink examples.
>>
>> I set the required properties (Resources, AM ContainerLaunchContext etc.) on
>> the YARN client interface. What happens is the JobManager and TaskManager
>> processes start and based on the logs containers are running but the actual
>> application does not start. I'm probably missing the proper way to pass
>> parameters to the ApplicationMaster and it cannot pick up the application it
>> needs to run. Anyone knows where I could get some info on how to pass
>> runtime params to the AppMaster?
>>
>> The ApplicationMaster launchcontainer script includes the following:
>> exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx1024M
>> org.apache.flink.yarn.ApplicationMaster  -c
>> org.apache.flink.examples.java.graph.ConnectedComponents 1>
>> /tmp/stdOut5237161854714899800 2>  /tmp/stdErr606502839107545371 "
>>
>> Thank you,
>> Theofilos
>>