flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vinay patil <vinay18.pa...@gmail.com>
Subject Re: add FLINK_LIB_DIR to classpath on yarn -> add properties file to class path on yarn
Date Fri, 18 Nov 2016 11:20:04 GMT
Hi Max,

When I provided the config file using --yarnship option , it worked. Thank
you.

However I did not get this : " *it's better to add it to the ship files in
the *
*cluster descriptor*" Can you please tell me how to do this ?

Regards,
Vinay Patil

On Fri, Nov 18, 2016 at 4:12 AM, Maximilian Michels [via Apache Flink User
Mailing List archive.] <ml-node+s2336050n10187h49@n4.nabble.com> wrote:

> The JVM only accepts Jar files in the classpath. You will have to load
> your custom files from the working directory of the node where the lib
> directory is shipped.
>
> By the way, the /lib directory is meant for Jar files. If you want to
> ship a custom file, it's better to add it to the ship files in the
> cluster descriptor or use the --yarnship option from the command-line.
>
> -Max
>
>
> On Thu, Nov 17, 2016 at 5:37 AM, vinay patil <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=10187&i=0>> wrote:
>
> > Hi Robert,
> >
> > Sorry for the late response.
> >
> > I checked the TM and JM logs but am not able to see that file added in
> the
> > classpath.
> >
> > I have deployed Flink on EMR, and placed the yaml file in
> /usr/lib/flink/lib
> > folder.
> >
> > What am I doing wrong here ?
> >
> > Regards,
> > Vinay Patil
> >
> > On Wed, Oct 26, 2016 at 1:25 PM, rmetzger0 [via Apache Flink User
> Mailing
> > List archive.] <[hidden email]> wrote:
> >>
> >> Hi Vinay,
> >>
> >> the JobManager and TaskManager logs contain the classpath used when
> >> starting a container on YARN. Can you check if the yaml file is in the
> >> classpath?
> >>
> >> On Tue, Oct 25, 2016 at 8:28 AM, vinay patil <[hidden email]> wrote:
> >>>
> >>> Hi Max,
> >>>
> >>> As discussed here , I have put my yaml file in the flink lib
> directory,
> >>> but
> >>> still I am not able to get this file from classpath.
> >>>
> >>> I am using Flink 1.1.1 and cfg4j to load the file from classpath.
> >>>
> >>> Running the job on YARN in EMR using the below command:
> >>>
> >>> ./bin/flink run <jar_name>
> >>>
> >>> Can you please let me know what I am doing wrong here.
> >>>
> >>> Regards,
> >>> Vinay Patil
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> >>> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p9702.html
> >>> Sent from the Apache Flink User Mailing List archive. mailing list
> >>> archive at Nabble.com.
> >>
> >>
> >>
> >>
> >> ________________________________
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >>
> >> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p9719.html
> >> To start a new topic under Apache Flink User Mailing List archive.,
> email
> >> [hidden email]
> >> To unsubscribe from Apache Flink User Mailing List archive., click
> here.
> >> NAML
> >
> >
> >
> > ________________________________
> > View this message in context: Re: add FLINK_LIB_DIR to classpath on yarn
> ->
> > add properties file to class path on yarn
> >
> > Sent from the Apache Flink User Mailing List archive. mailing list
> archive
> > at Nabble.com.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p10187.html
> To start a new topic under Apache Flink User Mailing List archive., email
> ml-node+s2336050n1h83@n4.nabble.com
> To unsubscribe from Apache Flink User Mailing List archive., click here
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>
> .
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/add-FLINK-LIB-DIR-to-classpath-on-yarn-tp8097p10206.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Mime
View raw message