spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anfernee Xu <>
Subject Application not found in Spark historyserver in yarn-client mode
Date Wed, 14 Oct 2015 23:20:12 GMT

Here's the problem I'm facing, I have a standalone java application which
is periodically submit Spark jobs to my yarn cluster, btw I'm not using
'spark-submit' or 'org.apache.spark.launcher' to submit my jobs. These jobs
are successful and I can see them on Yarn RM webUI, but when I want to
follow the link to the app history, I always got 404(application is not
found) from Spark historyserver.

My code looks likes as below

    SparkConf conf = new
        .setJars(new String[]{IOUtil.getJar(MySparkApp.class)});

    conf.set("spark.yarn.historyServer.address", "");

    JavaSparkContext sc = new JavaSparkContext(conf);

    try {

     ... my application code




View raw message