spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Tretyakov <vladimir.tretya...@sematext.com>
Subject Spark REST API YARN client mode is not full?
Date Thu, 06 Oct 2016 12:40:30 GMT
Hi,

When I start Spark  v1.6 (cdh5.8.0) in Yarn client mode I see that 4040
port is avaiable, but UI shows nothing and API returns not full information.

I started Spark application like this:

spark-submit     --master yarn-client     --class
org.apache.spark.examples.SparkPi
/usr/lib/spark/examples/lib/spark-examples-1.6.0-cdh5.8.0-hadoop2.6.0-cdh5.8.0.jar
    10000

API returns me:

http://localhost:4040/api/v1/applications

[ {
  "name" : "Spark Pi",
  "attempts" : [ {
    "startTime" : "2016-10-05T11:27:54.558GMT",
    "endTime" : "1969-12-31T23:59:59.999GMT",
    "sparkUser" : "",
    "completed" : false
  } ]
} ]

Where is application id? How I can get more detailed information about
application without this id (I am talking about /applications/[app-id]/jobs,
/applications/[app-id]/stages etc urls from
http://spark.apache.org/docs/1.6.0/monitoring.html)?

UI also shows me empty pages.

Without appId we cannot use other REST API calls. Is there any other way to
get RUNNING application ids?

Please help me understand what's going on.

Mime
View raw message