Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 283FE200C50 for ; Sat, 8 Apr 2017 12:14:05 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 26F07160B96; Sat, 8 Apr 2017 10:14:05 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 6E253160B88 for ; Sat, 8 Apr 2017 12:14:04 +0200 (CEST) Received: (qmail 87960 invoked by uid 500); 8 Apr 2017 10:14:03 -0000 Mailing-List: contact commits-help@zeppelin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zeppelin.apache.org Delivered-To: mailing list commits@zeppelin.apache.org Received: (qmail 87951 invoked by uid 99); 8 Apr 2017 10:14:03 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 08 Apr 2017 10:14:03 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 4D7B2E024D; Sat, 8 Apr 2017 10:14:03 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: zjffdu@apache.org To: commits@zeppelin.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: zeppelin git commit: ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master Date: Sat, 8 Apr 2017 10:14:03 +0000 (UTC) archived-at: Sat, 08 Apr 2017 10:14:05 -0000 Repository: zeppelin Updated Branches: refs/heads/branch-0.7 d0b4287d5 -> 46b196575 ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master `SparkUI.addUIAddress` is removed in spark master which cause this error. Actually spark 2.0 introduce new api `SparkContext.uiWebUrl` (SPARK-14576) which is public in contrast `SparkUI.addUIAddress` is private. This PR would use `SparkUI.addUIAddress` for spark 1 and `SparkContext.uiWebUrl` for spark 2. [Improvement | Refactoring] * [ ] - Task * https://issues.apache.org/jira/browse/ZEPPELIN-2150 Manually verify it on spark 1.6.2, spark 2.1.0 and spark master * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang Closes #2231 from zjffdu/ZEPPELIN-2150 and squashes the following commits: 499b623 [Jeff Zhang] return null when sparkUIUrl is not avaialbe e285278 [Jeff Zhang] ZEPPELIN-2150. NoSuchMethodError: org.apache.spark.ui.SparkUI.appUIAddress() for the spark master (cherry picked from commit 495be1ede1c0f5fc949275153dcaa1c6de5ddcdd) Signed-off-by: Jeff Zhang Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/46b19657 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/46b19657 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/46b19657 Branch: refs/heads/branch-0.7 Commit: 46b1965752522f255e34002dbd9288cca1495b44 Parents: d0b4287 Author: Jeff Zhang Authored: Sat Apr 8 08:15:14 2017 +0800 Committer: Jeff Zhang Committed: Sat Apr 8 18:12:33 2017 +0800 ---------------------------------------------------------------------- .../apache/zeppelin/spark/SparkInterpreter.java | 22 +++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/46b19657/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java ---------------------------------------------------------------------- diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java index 47f8080..a2eeac6 100644 --- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java +++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java @@ -961,11 +961,23 @@ public class SparkInterpreter extends Interpreter { numReferenceOfSparkContext.incrementAndGet(); } - private String getSparkUIUrl() { - Option sparkUiOption = (Option) Utils.invokeMethod(sc, "ui"); - SparkUI sparkUi = sparkUiOption.get(); - String sparkWebUrl = sparkUi.appUIAddress(); - return sparkWebUrl; + public String getSparkUIUrl() { + if (sparkUrl != null) { + return sparkUrl; + } + + if (sparkVersion.newerThanEquals(SparkVersion.SPARK_2_0_0)) { + Option uiWebUrlOption = (Option) Utils.invokeMethod(sc, "uiWebUrl"); + if (uiWebUrlOption.isDefined()) { + return uiWebUrlOption.get(); + } + } else { + Option sparkUIOption = (Option) Utils.invokeMethod(sc, "ui"); + if (sparkUIOption.isDefined()) { + return (String) Utils.invokeMethod(sparkUIOption.get(), "appUIAddress"); + } + } + return null; } private Results.Result interpret(String line) {